Sep 19 12:25:14.286770 ip-10-0-136-161 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 12:25:17.526982 ip-10-0-136-161 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:17.526982 ip-10-0-136-161 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 12:25:17.526982 ip-10-0-136-161 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:17.526982 ip-10-0-136-161 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 12:25:17.526982 ip-10-0-136-161 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:17.527761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.527059    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 12:25:17.530111 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530092    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.530111 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530110    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530114    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530118    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530121    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530124    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530127    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530129    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530132    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530135    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530138    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530141    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530144    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530147    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530150    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530153    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530155    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530158    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530168    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530171    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.530176 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530173    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530176    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530179    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530182    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530185    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530188    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530193    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530196    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530200    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530202    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530205    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530208    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530210    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530213    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530216    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530218    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530221    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530223    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530226    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.530633 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530228    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530231    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530233    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530236    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530239    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530241    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530243    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530246    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530248    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530251    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530255    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530258    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530261    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530263    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530266    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530269    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530272    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530275    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530277    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530280    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.531090 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530283    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530286    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530288    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530291    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530293    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530296    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530298    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530301    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530303    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530306    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530309    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530311    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530314    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530317    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530319    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530322    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530325    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530327    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530329    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530332    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.531586 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530334    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530337    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530340    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530342    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530345    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530347    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530350    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530354    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530357    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530360    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530362    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530365    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530367    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530370    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530374    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530378    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530381    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530384    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530387    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.530390    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.532126 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531096    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531103    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531106    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531110    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531114    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531118    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531120    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531123    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531126    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531128    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531131    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531133    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531136    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531139    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531142    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531144    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531147    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531151    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531155    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.532641 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531159    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531162    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531165    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531168    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531170    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531173    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531176    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531179    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531182    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531184    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531187    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531189    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531193    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531195    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531197    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531200    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531202    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531206    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531208    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.533128 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531211    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531214    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531216    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531219    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531221    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531224    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531226    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531229    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531231    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531234    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531236    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531239    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531241    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531244    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531247    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531249    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531252    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531254    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531256    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531259    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.533625 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531261    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531264    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531266    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531269    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531271    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531274    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531276    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531280    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531283    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531285    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531289    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531292    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531294    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531302    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531305    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531308    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531310    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531313    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531315    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531318    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.534105 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531320    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531323    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531326    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531328    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531332    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531335    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531337    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531340    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531343    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531345    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531348    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531350    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531353    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531355    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531358    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531361    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531363    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531366    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531368    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531371    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.534587 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531373    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531820    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531828    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531832    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531836    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531839    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531842    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531846    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531852    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531856    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531858    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531861    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531864    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531867    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531870    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531872    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531875    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531878    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531880    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531883    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.535068 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531886    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531888    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531892    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531896    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531899    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531902    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531905    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531908    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531911    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531913    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531916    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531919    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531921    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531924    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531932    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531934    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531937    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531939    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531943    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531945    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.535561 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531949    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531952    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531955    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531958    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531961    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531964    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531966    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531969    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531972    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531974    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531977    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531980    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531982    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531985    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531988    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531991    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531994    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531996    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.531999    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532002    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.536077 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532004    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532007    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532009    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532012    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532015    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532017    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532021    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532024    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532027    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532029    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532032    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532035    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532038    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532041    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532044    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532046    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532050    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532054    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532057    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532060    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.536576 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532063    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532066    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532068    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532071    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532074    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532076    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532079    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532081    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532084    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532087    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532089    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532092    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532095    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532097    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532100    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532103    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532107    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532110    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532114    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.537102 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532117    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532197    2568 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532247    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532254    2568 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532258    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532265    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532269    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532274    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532278    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532281    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532285    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532288    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532291    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532295    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532298    2568 flags.go:64] FLAG: --cgroup-root=""
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532301    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532304    2568 flags.go:64] FLAG: --client-ca-file=""
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532306    2568 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532310    2568 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532315    2568 flags.go:64] FLAG: --cluster-domain=""
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532318    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532321    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532324    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532327    2568 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 12:25:17.537554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532331    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532335    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532338    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532341    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532345    2568 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532348    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532351    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532355    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532358    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532362    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532365    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532368    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532371    2568 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532376    2568 flags.go:64] FLAG: --enable-server="true"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532379    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532384    2568 flags.go:64] FLAG: --event-burst="100"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532387    2568 flags.go:64] FLAG: --event-qps="50"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532390    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532393    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532396    2568 flags.go:64] FLAG: --eviction-hard=""
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532405    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532407    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532410    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532413    2568 flags.go:64] FLAG: --eviction-soft=""
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532416    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 12:25:17.538126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532419    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532423    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532426    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532429    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532432    2568 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532434    2568 flags.go:64] FLAG: --feature-gates=""
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532438    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532441    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532444    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532447    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532451    2568 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532454    2568 flags.go:64] FLAG: --help="false"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532457    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532460    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532464    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532467    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532470    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532474    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532477    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532480    2568 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532484    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532487    2568 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532490    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532505    2568 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 12:25:17.538738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532508    2568 flags.go:64] FLAG: --kube-reserved=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532511    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532514    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532517    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532520    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532523    2568 flags.go:64] FLAG: --lock-file=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532526    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532529    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532532    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532538    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532541    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532544    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532547    2568 flags.go:64] FLAG: --logging-format="text"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532549    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532552    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532555    2568 flags.go:64] FLAG: --manifest-url=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532558    2568 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532563    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532567    2568 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532571    2568 flags.go:64] FLAG: --max-pods="110"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532574    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532577    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532581    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532584    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532587    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 12:25:17.539349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532591    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532594    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532602    2568 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532606    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532609    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532612    2568 flags.go:64] FLAG: --pod-cidr=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532616    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532622    2568 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532625    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532629    2568 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532632    2568 flags.go:64] FLAG: --port="10250"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532635    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532638    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08937b4f38ea6ae8f"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532641    2568 flags.go:64] FLAG: --qos-reserved=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532645    2568 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532648    2568 flags.go:64] FLAG: --register-node="true"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532651    2568 flags.go:64] FLAG: --register-with-taints=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532655    2568 flags.go:64] FLAG: --registry-burst="10"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532658    2568 flags.go:64] FLAG: --registry-qps="5"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532661    2568 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532664    2568 flags.go:64] FLAG: --reserved-memory=""
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532668    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532671    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532674    2568 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532677    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 12:25:17.540022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532680    2568 flags.go:64] FLAG: --runonce="false"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532683    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532686    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532690    2568 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532694    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532697    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532700    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532704    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532707    2568 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532710    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532712    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532716    2568 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532719    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532722    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532726    2568 flags.go:64] FLAG: --system-cgroups=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532729    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532734    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532737    2568 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532740    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532746    2568 flags.go:64] FLAG: --tls-min-version=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532749    2568 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532752    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532754    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532757    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532761    2568 flags.go:64] FLAG: --v="2"
Sep 19 12:25:17.540695 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532765    2568 flags.go:64] FLAG: --version="false"
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532769    2568 flags.go:64] FLAG: --vmodule=""
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532773    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.532776    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532913    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532917    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532920    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532923    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532926    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532929    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532932    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532937    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532939    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532942    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532945    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532948    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532950    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532953    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532956    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532959    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.541304 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532962    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532964    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532968    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532970    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532973    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532976    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532978    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532993    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532995    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.532998    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533001    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533003    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533007    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533010    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533013    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533016    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533018    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533021    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533023    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.541799 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533027    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533030    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533034    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533036    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533041    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533044    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533047    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533050    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533053    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533055    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533058    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533061    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533063    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533066    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533068    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533071    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533074    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533077    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533079    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.542256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533082    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533084    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533086    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533089    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533092    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533094    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533097    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533099    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533102    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533104    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533107    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533109    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533112    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533114    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533117    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533119    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533122    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533126    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533128    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533131    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.542732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533134    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533137    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533141    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533144    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533147    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533150    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533152    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533155    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533158    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533160    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533163    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533166    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533169    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533171    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533174    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533176    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533179    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533181    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533184    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533186    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.543214 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533189    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533191    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533193    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533196    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.533198    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.533920    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.541575    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.541592    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.541611    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.541620    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541686    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541690    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541693    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541696    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.543719 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541698    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541702    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541706    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541709    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541712    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541715    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541717    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541719    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541722    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541725    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541727    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541730    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541732    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541735    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541738    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541740    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541743    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541746    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541748    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541751    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.544140 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541753    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541756    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541758    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541761    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541764    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541773    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541776    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541778    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541781    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541784    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541786    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541789    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541791    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541794    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541797    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541799    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541802    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541805    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541808    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541810    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.544645 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541813    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541816    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541818    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541821    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541823    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541826    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541828    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541831    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541833    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541836    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541839    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541841    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541844    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541847    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541850    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541852    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541855    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541858    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541867    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541870    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.545178 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541873    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541877    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541880    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541883    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541886    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541889    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541892    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541894    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541897    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541900    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541902    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541905    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541907    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541910    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541912    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541915    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541918    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541920    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541923    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.545689 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541925    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541928    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541931    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541933    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541938    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541941    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541944    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541947    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541950    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541953    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541955    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541957    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541965    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541968    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541970    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.546142 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.541973    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.541978    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542126    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542131    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542134    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542136    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542139    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542141    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542144    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542146    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542149    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542151    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542154    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542157    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542159    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:17.546591 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542162    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542165    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542167    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542170    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542172    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542175    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542178    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542181    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542184    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542187    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542189    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542192    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542194    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542196    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542199    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542207    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542210    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542212    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542214    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:17.547023 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542217    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542219    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542222    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542224    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542227    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542229    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542231    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542234    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542236    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542239    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542243    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542246    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542249    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542252    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542254    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542257    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542260    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542262    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542265    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:17.547746 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542268    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542271    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542274    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542277    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542279    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542282    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542284    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542287    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542289    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542292    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542300    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542303    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542305    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542308    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542310    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542313    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542315    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542319    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542322    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:17.548387 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542325    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542327    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542330    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542333    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542336    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542339    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542341    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542344    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542346    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542349    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542351    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542354    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542356    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542359    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542361    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542365    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542369    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542371    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542374    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542376    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:17.548892 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542379    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542381    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542384    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542386    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542395    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542398    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542400    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542402    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:17.542405    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.542410    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.542555    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.544777    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.545531    2568 server.go:1019] "Starting client certificate rotation"
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.545629    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 12:25:17.549389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.545672    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 12:25:17.549745 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.548863    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 12:25:17.550661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.550645    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 12:25:17.554664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.554646    2568 log.go:25] "Validated CRI v1 runtime API"
Sep 19 12:25:17.556433 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.556417    2568 log.go:25] "Validated CRI v1 image API"
Sep 19 12:25:17.557250 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.557233    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 12:25:17.559963 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.559936    2568 fs.go:135] Filesystem UUIDs: map[13a92478-de89-411b-b424-04432207c7c8:/dev/nvme0n1p4 6d713efd-4fbf-4f16-b5e6-04942c275c20:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 19 12:25:17.560026 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.559960    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 12:25:17.566254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.566116    2568 manager.go:216] Machine: {Timestamp:2026-09-19 12:25:17.565298025 +0000 UTC m=+1.854853788 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3195699 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec223a1a34b76c84b24d62f9388beb19 SystemUUID:ec223a1a-34b7-6c84-b24d-62f9388beb19 BootID:5b638fcd-3725-42da-a84c-af0e1a80c1ea Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:eb:b7:2e:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:eb:b7:2e:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:06:98:07:1d:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 12:25:17.566254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.566251    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 12:25:17.566406 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.566351    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 12:25:17.568024 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.567995    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 12:25:17.568191 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.568027    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 12:25:17.569183 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.569170    2568 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 12:25:17.569245 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.569191    2568 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 12:25:17.569245 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.569201    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 12:25:17.569245 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.569217    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 12:25:17.569368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.569300    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 12:25:17.573043 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573030    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 12:25:17.573175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573164    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 12:25:17.573238 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573212    2568 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 12:25:17.573238 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573236    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 12:25:17.573327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573251    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 12:25:17.573327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573264    2568 kubelet.go:398] "Adding apiserver pod source"
Sep 19 12:25:17.573327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573285    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 12:25:17.573660 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573646    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 12:25:17.573733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.573669    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 12:25:17.577377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.577351    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 12:25:17.580206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.580181    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 12:25:17.580749 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.580733    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 12:25:17.580810 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.580765    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 12:25:17.581800 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581790    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581804    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581810    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581816    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581825    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581830    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581844    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 12:25:17.581848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581850    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581856    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581861    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581869    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581874    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581893    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 12:25:17.582042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.581898    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 12:25:17.585483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.585456    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 12:25:17.585848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.585837    2568 server.go:1268] "Started kubelet"
Sep 19 12:25:17.585966 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.585920    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 12:25:17.586091 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.586058    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 12:25:17.586091 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.586061    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 12:25:17.586091 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.586049    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 12:25:17.586281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.586121    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 12:25:17.586667 ip-10-0-136-161 systemd[1]: Started Kubernetes Kubelet.
Sep 19 12:25:17.587915 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.587729    2568 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 12:25:17.589731 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.589698    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 12:25:17.590221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.590208    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 12:25:17.596292 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.596278    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 12:25:17.596355 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.596302    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 12:25:17.596403 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.596362    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.597048 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597033    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 12:25:17.597204 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597036    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 12:25:17.597204 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597205    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 12:25:17.597338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597323    2568 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 12:25:17.597338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597331    2568 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 12:25:17.597453 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597440    2568 factory.go:153] Registering CRI-O factory
Sep 19 12:25:17.597487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597458    2568 factory.go:223] Registration of the crio container factory successfully
Sep 19 12:25:17.597540 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597530    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 12:25:17.597591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597539    2568 factory.go:55] Registering systemd factory
Sep 19 12:25:17.597591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597547    2568 factory.go:223] Registration of the systemd container factory successfully
Sep 19 12:25:17.597591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597561    2568 factory.go:103] Registering Raw factory
Sep 19 12:25:17.597591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597570    2568 manager.go:1196] Started watching for new ooms in manager
Sep 19 12:25:17.598017 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.597998    2568 manager.go:349] Starting recovery of all containers
Sep 19 12:25:17.598231 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.598208    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 12:25:17.598300 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.598282    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 12:25:17.599659 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.599635    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pnmj2"
Sep 19 12:25:17.618904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.618880    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pnmj2"
Sep 19 12:25:17.621705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.621686    2568 manager.go:354] Recovery completed
Sep 19 12:25:17.626487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.626471    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.628278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628260    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.628349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628291    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.628349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628305    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.628838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628825    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 12:25:17.628886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628838    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 12:25:17.628886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628852    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 12:25:17.628964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.628831    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.629759 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629744    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.629829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629763    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.629829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629772    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.629994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629968    2568 policy_none.go:49] "None policy: Start"
Sep 19 12:25:17.629994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629985    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 12:25:17.629994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.629994    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 12:25:17.630797 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.630786    2568 policy_none.go:47] "Start"
Sep 19 12:25:17.671089 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671075    2568 manager.go:337] "Starting Device Plugin manager"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.671111    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671122    2568 server.go:87] "Starting device plugin registration server"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671386    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671406    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671536    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671606    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.671614    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.672062    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 12:25:17.687733 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.672114    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.714178 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.714142    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 12:25:17.715342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.715323    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 12:25:17.715429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.715365    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 12:25:17.715429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.715385    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 12:25:17.715512 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.715423    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 12:25:17.717556 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.717540    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:17.772369 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.772341    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.773247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.773231    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.773361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.773259    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.773361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.773268    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.773361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.773290    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.782391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.782332    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.782391 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.782354    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-161.ec2.internal\": node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.798551 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.798533    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.815541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.815517    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"]
Sep 19 12:25:17.815642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.815586    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.816554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.816539    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.816630 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.816567    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.816630 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.816577    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.818606 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.818593    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.818739 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.818722    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.818803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.818790    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.819290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819275    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.819372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819296    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.819372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819306    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.819372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819295    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.819473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819386    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.819473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.819395    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.821177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.821164    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.821221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.821187    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:17.821829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.821816    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:17.821909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.821851    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:17.821909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.821867    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:17.844111 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.844092    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-161.ec2.internal\" not found" node="ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.848334 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.848321    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-161.ec2.internal\" not found" node="ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.899293 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.899274    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.899393 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.899347    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.899393 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.899374    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.899459 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.899397    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/657c4b0f272c5c6a99d291261947b4fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-161.ec2.internal\" (UID: \"657c4b0f272c5c6a99d291261947b4fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999731 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:17.999692    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:17.999731 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999755    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999781    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/657c4b0f272c5c6a99d291261947b4fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-161.ec2.internal\" (UID: \"657c4b0f272c5c6a99d291261947b4fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999821    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/754257620d708c54c6031bfd4f611893-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal\" (UID: \"754257620d708c54c6031bfd4f611893\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:17.999908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:17.999826    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/657c4b0f272c5c6a99d291261947b4fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-161.ec2.internal\" (UID: \"657c4b0f272c5c6a99d291261947b4fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:18.100448 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.100367    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.148556 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.148532    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:18.152378 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.152358    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:18.201139 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.201102    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.301436 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.301408    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.401890 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.401856    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.502367 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.502325    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.546656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.546487    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 12:25:18.547104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.546810    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 12:25:18.602583 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.602551    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.624336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.624303    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 12:20:17 +0000 UTC" deadline="2028-03-01 11:01:23.486447718 +0000 UTC"
Sep 19 12:25:18.624336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.624331    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12694h36m4.862119684s"
Sep 19 12:25:18.632967 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.632939    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:18.678970 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:18.678940    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod754257620d708c54c6031bfd4f611893.slice/crio-ccef26437c439eabbf26bdb8ac9ca04f689bf1d27fe6cffa4d919e458120ba0d WatchSource:0}: Error finding container ccef26437c439eabbf26bdb8ac9ca04f689bf1d27fe6cffa4d919e458120ba0d: Status 404 returned error can't find the container with id ccef26437c439eabbf26bdb8ac9ca04f689bf1d27fe6cffa4d919e458120ba0d
Sep 19 12:25:18.683431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.683418    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:25:18.703285 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:18.703260    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-161.ec2.internal\" not found"
Sep 19 12:25:18.718844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.718808    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal" event={"ID":"754257620d708c54c6031bfd4f611893","Type":"ContainerStarted","Data":"ccef26437c439eabbf26bdb8ac9ca04f689bf1d27fe6cffa4d919e458120ba0d"}
Sep 19 12:25:18.739683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.739665    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:18.754792 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:18.754771    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657c4b0f272c5c6a99d291261947b4fd.slice/crio-d118d5b0b17aa0a87698b23e5f1a34afabdb38b20dc5265b724fc000fbfd7eb1 WatchSource:0}: Error finding container d118d5b0b17aa0a87698b23e5f1a34afabdb38b20dc5265b724fc000fbfd7eb1: Status 404 returned error can't find the container with id d118d5b0b17aa0a87698b23e5f1a34afabdb38b20dc5265b724fc000fbfd7eb1
Sep 19 12:25:18.797147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.797116    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:18.812397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.812373    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 12:25:18.813397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.813385    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal"
Sep 19 12:25:18.824896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:18.824880    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 12:25:19.097332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.097253    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:19.574462 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.574427    2568 apiserver.go:52] "Watching apiserver"
Sep 19 12:25:19.580665 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.580639    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 12:25:19.581097 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.581076    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-6cd6x","openshift-network-diagnostics/network-check-target-l55mc","openshift-ovn-kubernetes/ovnkube-node-nhggd","kube-system/global-pull-secret-syncer-658s9","kube-system/konnectivity-agent-sbsbv","openshift-cluster-node-tuning-operator/tuned-p2ptk","openshift-dns/node-resolver-8v6pn","openshift-image-registry/node-ca-vnjc7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal","openshift-multus/multus-s7xlq","openshift-network-operator/iptables-alerter-nbjpc","kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx","openshift-multus/multus-additional-cni-plugins-qt8cr"]
Sep 19 12:25:19.583838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.583814    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.586032 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.586012    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.586132 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.586105    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m8pst\""
Sep 19 12:25:19.586294 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.586276    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.586554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.586536    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.588338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.588316    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.588674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.588653    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 12:25:19.588894 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.588876    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.588977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.588950    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9k2xd\""
Sep 19 12:25:19.589066 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.588877    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.590573 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.590548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.590671 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.590614    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:19.591138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591121    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.591200 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rp2g4\""
Sep 19 12:25:19.591259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591195    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 12:25:19.591259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591205    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 12:25:19.591573 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591556    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.591633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591558    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 12:25:19.591693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.591564    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 12:25:19.594832 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.594444    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.596445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.596420    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 12:25:19.596553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.596491    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 12:25:19.596664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.596650    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 12:25:19.596724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.596713    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jmmh4\""
Sep 19 12:25:19.596777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.596758    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.599216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.598961    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.599624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.599484    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.599733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.599656    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ghd89\""
Sep 19 12:25:19.599733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.599655    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.600931 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.600911    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.601038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.600942    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 12:25:19.601200 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.601150    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 12:25:19.601251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.601210    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 12:25:19.601468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.601442    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.601637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.601616    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m8g88\""
Sep 19 12:25:19.602130 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.601853    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.602130 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.601911    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:19.605990 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.605970    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:19.606090 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.606033    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:19.606090 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.606051    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 12:25:19.608702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.608682    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.609122 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609101    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-systemd-units\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-kubelet-config\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.609216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609167    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbsd\" (UniqueName: \"kubernetes.io/projected/4e665128-4211-4348-a845-ac40c87dd495-kube-api-access-xwbsd\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.609216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609189    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609235    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aee997a9-8dd4-4ff1-897f-1850eec78e94-agent-certs\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609256    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-bin\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609322    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7495\" (UniqueName: \"kubernetes.io/projected/049d77b8-c91f-48ce-badc-f9b904873b7f-kube-api-access-t7495\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-var-lib-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609367    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-ovn\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609390    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-conf\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609410    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-run\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609431    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-host\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609451    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-device-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609473    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-netns\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609509    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-config\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-script-lib\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f9ww\" (UniqueName: \"kubernetes.io/projected/b122ea00-99d3-47a3-b170-cf888a0ff9af-kube-api-access-4f9ww\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-modprobe-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609610    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609645    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-registration-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-env-overrides\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-slash\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-etc-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.609747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-socket-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-sys-fs\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b15a0a7b-c722-403f-a655-48d1961671c1-hosts-file\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-dbus\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-kubelet\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-log-socket\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609880    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609893    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-netd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609909    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-etc-selinux\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-system-cni-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15a0a7b-c722-403f-a655-48d1961671c1-tmp-dir\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609967    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-systemd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.609989    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysconfig\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-systemd\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610034    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-var-lib-kubelet\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.610431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610055    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-tuned\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610081    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aee997a9-8dd4-4ff1-897f-1850eec78e94-konnectivity-ca\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610127    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-node-log\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovn-node-metrics-cert\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610193    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-cnibin\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610216    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-kubernetes\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610238    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-lib-modules\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610259    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-tmp\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610281    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-sys\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610305    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmkm\" (UniqueName: \"kubernetes.io/projected/bb945c06-675c-4aa3-9c17-37ab108b9b03-kube-api-access-nbmkm\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjpg\" (UniqueName: \"kubernetes.io/projected/b15a0a7b-c722-403f-a655-48d1961671c1-kube-api-access-4qjpg\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610700    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 12:25:19.611139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.610715    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8wf5x\""
Sep 19 12:25:19.612584 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.612468    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.614429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.614409    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.614938 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.614919    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.615024 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.614974    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 12:25:19.615175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.615156    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2g2rt\""
Sep 19 12:25:19.615175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.615171    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.617290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.617271    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 12:25:19.617375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.617284    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 12:25:19.617375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.617364    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d9vjp\""
Sep 19 12:25:19.617483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.617467    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 12:25:19.630226 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.630053    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tttd8"
Sep 19 12:25:19.637580 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.637555    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tttd8"
Sep 19 12:25:19.698066 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.698042    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-bin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-device-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-netns\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711538    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-config\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711564    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-device-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711573    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-script-lib\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.711601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f9ww\" (UniqueName: \"kubernetes.io/projected/b122ea00-99d3-47a3-b170-cf888a0ff9af-kube-api-access-4f9ww\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-config\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-modprobe-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712433    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-serviceca\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712467    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-cni-binary-copy\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovnkube-script-lib\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712553    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-socket-dir-parent\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712652    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-netns\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.713804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-multus-daemon-config\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712755    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-modprobe-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-registration-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-env-overrides\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-slash\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712948    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-etc-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.712983    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-k8s-cni-cncf-io\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g2r2\" (UniqueName: \"kubernetes.io/projected/12fbbda2-9c99-4465-925a-0a834053447c-kube-api-access-8g2r2\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-socket-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-sys-fs\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713131    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bkgp\" (UniqueName: \"kubernetes.io/projected/e280c2b4-e6d5-4248-b857-9db63cd55f2c-kube-api-access-9bkgp\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713193    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-multus\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713250    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b15a0a7b-c722-403f-a655-48d1961671c1-hosts-file\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-cnibin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.714768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713323    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-dbus\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713342    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713352    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-kubelet\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713385    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-log-socket\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713431    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713455    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-socket-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713461    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-netd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713539    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-netd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713575    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-etc-selinux\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713608    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-system-cni-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713617    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713633    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15a0a7b-c722-403f-a655-48d1961671c1-tmp-dir\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713677    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-registration-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-systemd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713741    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysconfig\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713803    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-kubelet\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713848    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysconfig\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.715439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713895    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-systemd\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713905    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-log-socket\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.713978    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-etc-selinux\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714057    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-sys-fs\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714119    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-system-cni-dir\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714146    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714181    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-slash\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-systemd\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b15a0a7b-c722-403f-a655-48d1961671c1-tmp-dir\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714248    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b122ea00-99d3-47a3-b170-cf888a0ff9af-env-overrides\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-systemd\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714326    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-var-lib-kubelet\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714387    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-tuned\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714401    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b15a0a7b-c722-403f-a655-48d1961671c1-hosts-file\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714443    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-host\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714458    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-var-lib-kubelet\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714459    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-dbus\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714472    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aee997a9-8dd4-4ff1-897f-1850eec78e94-konnectivity-ca\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.716146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-node-log\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovn-node-metrics-cert\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714645    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-cnibin\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714723    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-node-log\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714770    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714780    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714803    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-etc-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.714911    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-d\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715007    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-netns\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/049d77b8-c91f-48ce-badc-f9b904873b7f-cnibin\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-kubernetes\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715958    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-lib-modules\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715978    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aee997a9-8dd4-4ff1-897f-1850eec78e94-konnectivity-ca\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715988    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-kubernetes\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.715998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-tmp\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716005    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/049d77b8-c91f-48ce-badc-f9b904873b7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.717010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh625\" (UniqueName: \"kubernetes.io/projected/49b956e3-20ce-47e3-9c91-d7f4194127d3-kube-api-access-gh625\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-hostroot\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716103    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-sys\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716134    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmkm\" (UniqueName: \"kubernetes.io/projected/bb945c06-675c-4aa3-9c17-37ab108b9b03-kube-api-access-nbmkm\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjpg\" (UniqueName: \"kubernetes.io/projected/b15a0a7b-c722-403f-a655-48d1961671c1-kube-api-access-4qjpg\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-lib-modules\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-sys\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t48lh\" (UniqueName: \"kubernetes.io/projected/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-kube-api-access-t48lh\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716224    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-system-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716259    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-multus-certs\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716291    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e280c2b4-e6d5-4248-b857-9db63cd55f2c-iptables-alerter-script\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716314    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-systemd-units\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716335    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-kubelet-config\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbsd\" (UniqueName: \"kubernetes.io/projected/4e665128-4211-4348-a845-ac40c87dd495-kube-api-access-xwbsd\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716382    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-conf-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716514    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.717858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aee997a9-8dd4-4ff1-897f-1850eec78e94-agent-certs\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716644    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-bin\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716690    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.716698    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-kubelet\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.716790    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:20.216750403 +0000 UTC m=+4.506306159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716829    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-etc-kubernetes\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716856    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e280c2b4-e6d5-4248-b857-9db63cd55f2c-host-slash\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7495\" (UniqueName: \"kubernetes.io/projected/049d77b8-c91f-48ce-badc-f9b904873b7f-kube-api-access-t7495\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716905    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-var-lib-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716938    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-ovn\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-conf\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.716997    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-run\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717017    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-host\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717030    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e665128-4211-4348-a845-ac40c87dd495-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.719628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-systemd-units\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-run\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717137    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-host\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d9c3687-5584-4740-8d07-8cc36bb8213b-kubelet-config\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717182    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717285    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-sysctl-conf\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717319    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-var-lib-openvswitch\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717347    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-run-ovn\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717384    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-cni-bin\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.717425    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b122ea00-99d3-47a3-b170-cf888a0ff9af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.718766    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-tmp\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.718876    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b122ea00-99d3-47a3-b170-cf888a0ff9af-ovn-node-metrics-cert\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.719648    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f9ww\" (UniqueName: \"kubernetes.io/projected/b122ea00-99d3-47a3-b170-cf888a0ff9af-kube-api-access-4f9ww\") pod \"ovnkube-node-nhggd\" (UID: \"b122ea00-99d3-47a3-b170-cf888a0ff9af\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.720461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.719916    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aee997a9-8dd4-4ff1-897f-1850eec78e94-agent-certs\") pod \"konnectivity-agent-sbsbv\" (UID: \"aee997a9-8dd4-4ff1-897f-1850eec78e94\") " pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.721105 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.720958    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb945c06-675c-4aa3-9c17-37ab108b9b03-etc-tuned\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.721542 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.721515    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal" event={"ID":"657c4b0f272c5c6a99d291261947b4fd","Type":"ContainerStarted","Data":"d118d5b0b17aa0a87698b23e5f1a34afabdb38b20dc5265b724fc000fbfd7eb1"}
Sep 19 12:25:19.724138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.724104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjpg\" (UniqueName: \"kubernetes.io/projected/b15a0a7b-c722-403f-a655-48d1961671c1-kube-api-access-4qjpg\") pod \"node-resolver-8v6pn\" (UID: \"b15a0a7b-c722-403f-a655-48d1961671c1\") " pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.727359 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.727338    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:19.727887 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.727842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmkm\" (UniqueName: \"kubernetes.io/projected/bb945c06-675c-4aa3-9c17-37ab108b9b03-kube-api-access-nbmkm\") pod \"tuned-p2ptk\" (UID: \"bb945c06-675c-4aa3-9c17-37ab108b9b03\") " pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.728023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.728000    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbsd\" (UniqueName: \"kubernetes.io/projected/4e665128-4211-4348-a845-ac40c87dd495-kube-api-access-xwbsd\") pod \"aws-ebs-csi-driver-node-hmjbx\" (UID: \"4e665128-4211-4348-a845-ac40c87dd495\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.728760 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.728736    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7495\" (UniqueName: \"kubernetes.io/projected/049d77b8-c91f-48ce-badc-f9b904873b7f-kube-api-access-t7495\") pod \"multus-additional-cni-plugins-qt8cr\" (UID: \"049d77b8-c91f-48ce-badc-f9b904873b7f\") " pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.817558 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817520    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t48lh\" (UniqueName: \"kubernetes.io/projected/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-kube-api-access-t48lh\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.817747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817569    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-system-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817605    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-multus-certs\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e280c2b4-e6d5-4248-b857-9db63cd55f2c-iptables-alerter-script\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.817747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817710    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-multus-certs\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817721    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-system-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-conf-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817848    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-conf-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-kubelet\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-etc-kubernetes\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e280c2b4-e6d5-4248-b857-9db63cd55f2c-host-slash\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817934    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-bin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-serviceca\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-etc-kubernetes\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817969    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-kubelet\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.817994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.817986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e280c2b4-e6d5-4248-b857-9db63cd55f2c-host-slash\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818090    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-bin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818139    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-cni-dir\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-cni-binary-copy\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-socket-dir-parent\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818182    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e280c2b4-e6d5-4248-b857-9db63cd55f2c-iptables-alerter-script\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818190    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-multus-daemon-config\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818226    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-k8s-cni-cncf-io\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818237    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-multus-socket-dir-parent\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818249    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g2r2\" (UniqueName: \"kubernetes.io/projected/12fbbda2-9c99-4465-925a-0a834053447c-kube-api-access-8g2r2\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-k8s-cni-cncf-io\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818269    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818314    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bkgp\" (UniqueName: \"kubernetes.io/projected/e280c2b4-e6d5-4248-b857-9db63cd55f2c-kube-api-access-9bkgp\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-multus\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-cnibin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.818380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818381    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-serviceca\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-host\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818431    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-host\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-netns\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh625\" (UniqueName: \"kubernetes.io/projected/49b956e3-20ce-47e3-9c91-d7f4194127d3-kube-api-access-gh625\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-cnibin\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.818525    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818566    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-run-netns\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-hostroot\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.818582    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:20.3185664 +0000 UTC m=+4.608122148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818468    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-host-var-lib-cni-multus\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818663    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12fbbda2-9c99-4465-925a-0a834053447c-hostroot\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818676    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-cni-binary-copy\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.819053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.818737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12fbbda2-9c99-4465-925a-0a834053447c-multus-daemon-config\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.825382 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.825329    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:19.825382 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.825357    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:19.825382 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.825371    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:19.825604 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:19.825424    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:20.325407329 +0000 UTC m=+4.614963106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:19.825927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.825906    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t48lh\" (UniqueName: \"kubernetes.io/projected/bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5-kube-api-access-t48lh\") pod \"node-ca-vnjc7\" (UID: \"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5\") " pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:19.826884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.826866    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bkgp\" (UniqueName: \"kubernetes.io/projected/e280c2b4-e6d5-4248-b857-9db63cd55f2c-kube-api-access-9bkgp\") pod \"iptables-alerter-nbjpc\" (UID: \"e280c2b4-e6d5-4248-b857-9db63cd55f2c\") " pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.827553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.827533    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g2r2\" (UniqueName: \"kubernetes.io/projected/12fbbda2-9c99-4465-925a-0a834053447c-kube-api-access-8g2r2\") pod \"multus-s7xlq\" (UID: \"12fbbda2-9c99-4465-925a-0a834053447c\") " pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.827628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.827563    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh625\" (UniqueName: \"kubernetes.io/projected/49b956e3-20ce-47e3-9c91-d7f4194127d3-kube-api-access-gh625\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:19.897379 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.897344    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8v6pn"
Sep 19 12:25:19.907216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.907194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx"
Sep 19 12:25:19.918866 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.918842    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:19.924583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.924562    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:19.931023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.931004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p2ptk"
Sep 19 12:25:19.939583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.939567    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qt8cr"
Sep 19 12:25:19.947163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.947146    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s7xlq"
Sep 19 12:25:19.956633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.956614    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbjpc"
Sep 19 12:25:19.962086 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:19.962071    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vnjc7"
Sep 19 12:25:20.220962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.220929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:20.221183 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.221114    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:20.221254 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.221192    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.221172052 +0000 UTC m=+5.510727814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:20.321289 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.321248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:20.321455 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.321404    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:20.321519 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.321484    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.321465019 +0000 UTC m=+5.611020768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:20.421948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.421917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:20.422111 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.422069    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:20.422111 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.422088    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:20.422111 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.422098    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:20.422262 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:20.422149    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.422134707 +0000 UTC m=+5.711690477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:20.559807 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.559765    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode280c2b4_e6d5_4248_b857_9db63cd55f2c.slice/crio-519e46e238ba59d6311cd26eadd80189827cafba7179b65e476fe44b84f73055 WatchSource:0}: Error finding container 519e46e238ba59d6311cd26eadd80189827cafba7179b65e476fe44b84f73055: Status 404 returned error can't find the container with id 519e46e238ba59d6311cd26eadd80189827cafba7179b65e476fe44b84f73055
Sep 19 12:25:20.561620 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.561592    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049d77b8_c91f_48ce_badc_f9b904873b7f.slice/crio-2321b0b0aa19b85628b5f65c16d5b37fff2df32b6066dbc8865e24c4b7bd69e7 WatchSource:0}: Error finding container 2321b0b0aa19b85628b5f65c16d5b37fff2df32b6066dbc8865e24c4b7bd69e7: Status 404 returned error can't find the container with id 2321b0b0aa19b85628b5f65c16d5b37fff2df32b6066dbc8865e24c4b7bd69e7
Sep 19 12:25:20.563324 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.563302    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8fbf76_c9ce_4e7c_9629_5bc0c819bad5.slice/crio-b507f79eccf4fa2a5fb42c359a89eb5dc1a3d69ad42539732e6f63e37f4b3fa3 WatchSource:0}: Error finding container b507f79eccf4fa2a5fb42c359a89eb5dc1a3d69ad42539732e6f63e37f4b3fa3: Status 404 returned error can't find the container with id b507f79eccf4fa2a5fb42c359a89eb5dc1a3d69ad42539732e6f63e37f4b3fa3
Sep 19 12:25:20.567310 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.567286    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e665128_4211_4348_a845_ac40c87dd495.slice/crio-b4e5b781b5a83a799f4d3a5b4c0bc22a6c6cef79b59022e586fff2c26a23cd1b WatchSource:0}: Error finding container b4e5b781b5a83a799f4d3a5b4c0bc22a6c6cef79b59022e586fff2c26a23cd1b: Status 404 returned error can't find the container with id b4e5b781b5a83a799f4d3a5b4c0bc22a6c6cef79b59022e586fff2c26a23cd1b
Sep 19 12:25:20.568217 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.568195    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fbbda2_9c99_4465_925a_0a834053447c.slice/crio-33057d6599156e6f51d52edbebfdaad218588b50ef0fb8cb52e085120c12becf WatchSource:0}: Error finding container 33057d6599156e6f51d52edbebfdaad218588b50ef0fb8cb52e085120c12becf: Status 404 returned error can't find the container with id 33057d6599156e6f51d52edbebfdaad218588b50ef0fb8cb52e085120c12becf
Sep 19 12:25:20.590066 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.590046    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb945c06_675c_4aa3_9c17_37ab108b9b03.slice/crio-92bebaf54f593907c7348737fa2882aec15b1409425daa1de540f68ceeb85db9 WatchSource:0}: Error finding container 92bebaf54f593907c7348737fa2882aec15b1409425daa1de540f68ceeb85db9: Status 404 returned error can't find the container with id 92bebaf54f593907c7348737fa2882aec15b1409425daa1de540f68ceeb85db9
Sep 19 12:25:20.590697 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.590676    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb122ea00_99d3_47a3_b170_cf888a0ff9af.slice/crio-0b1391936859249a9d2f56320dba7b66f0dc0bfdbc71506cf613f377739d1286 WatchSource:0}: Error finding container 0b1391936859249a9d2f56320dba7b66f0dc0bfdbc71506cf613f377739d1286: Status 404 returned error can't find the container with id 0b1391936859249a9d2f56320dba7b66f0dc0bfdbc71506cf613f377739d1286
Sep 19 12:25:20.592063 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:20.592045    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15a0a7b_c722_403f_a655_48d1961671c1.slice/crio-98b82c8632e03d6e20b567fe70e3caf71f962624c1fda43985b8af860bd872e8 WatchSource:0}: Error finding container 98b82c8632e03d6e20b567fe70e3caf71f962624c1fda43985b8af860bd872e8: Status 404 returned error can't find the container with id 98b82c8632e03d6e20b567fe70e3caf71f962624c1fda43985b8af860bd872e8
Sep 19 12:25:20.641275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.641248    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 12:20:19 +0000 UTC" deadline="2028-05-24 13:26:39.845969701 +0000 UTC"
Sep 19 12:25:20.641275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.641274    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14713h1m19.204698957s"
Sep 19 12:25:20.724249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.724215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8v6pn" event={"ID":"b15a0a7b-c722-403f-a655-48d1961671c1","Type":"ContainerStarted","Data":"98b82c8632e03d6e20b567fe70e3caf71f962624c1fda43985b8af860bd872e8"}
Sep 19 12:25:20.725472 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.725448    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p2ptk" event={"ID":"bb945c06-675c-4aa3-9c17-37ab108b9b03","Type":"ContainerStarted","Data":"92bebaf54f593907c7348737fa2882aec15b1409425daa1de540f68ceeb85db9"}
Sep 19 12:25:20.726702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.726660    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s7xlq" event={"ID":"12fbbda2-9c99-4465-925a-0a834053447c","Type":"ContainerStarted","Data":"33057d6599156e6f51d52edbebfdaad218588b50ef0fb8cb52e085120c12becf"}
Sep 19 12:25:20.727826 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.727800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sbsbv" event={"ID":"aee997a9-8dd4-4ff1-897f-1850eec78e94","Type":"ContainerStarted","Data":"c58571b75b49a1e8153162da84f61cf80bda1c11f90ce55ee2313802225e312c"}
Sep 19 12:25:20.728871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.728855    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vnjc7" event={"ID":"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5","Type":"ContainerStarted","Data":"b507f79eccf4fa2a5fb42c359a89eb5dc1a3d69ad42539732e6f63e37f4b3fa3"}
Sep 19 12:25:20.729893 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.729870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerStarted","Data":"2321b0b0aa19b85628b5f65c16d5b37fff2df32b6066dbc8865e24c4b7bd69e7"}
Sep 19 12:25:20.731702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.731680    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal" event={"ID":"657c4b0f272c5c6a99d291261947b4fd","Type":"ContainerStarted","Data":"b77f2ab84b3e85505db162d389ae2d16a28da4577d9d0b7e040f52f9c99fa6c5"}
Sep 19 12:25:20.732809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.732789    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"0b1391936859249a9d2f56320dba7b66f0dc0bfdbc71506cf613f377739d1286"}
Sep 19 12:25:20.733800 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.733778    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx" event={"ID":"4e665128-4211-4348-a845-ac40c87dd495","Type":"ContainerStarted","Data":"b4e5b781b5a83a799f4d3a5b4c0bc22a6c6cef79b59022e586fff2c26a23cd1b"}
Sep 19 12:25:20.734995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.734975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbjpc" event={"ID":"e280c2b4-e6d5-4248-b857-9db63cd55f2c","Type":"ContainerStarted","Data":"519e46e238ba59d6311cd26eadd80189827cafba7179b65e476fe44b84f73055"}
Sep 19 12:25:20.745597 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:20.745560    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-161.ec2.internal" podStartSLOduration=2.745549408 podStartE2EDuration="2.745549408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:20.745237385 +0000 UTC m=+5.034793152" watchObservedRunningTime="2026-09-19 12:25:20.745549408 +0000 UTC m=+5.035105175"
Sep 19 12:25:21.227616 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.227571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:21.227885 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.227865    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:21.227955 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.227945    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:23.227926036 +0000 UTC m=+7.517481786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:21.328841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.328803    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:21.329005 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.328945    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:21.329086 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.329009    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:23.328990573 +0000 UTC m=+7.618546325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:21.430013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.429759    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:21.430199 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.430092    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:21.430199 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.430117    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:21.430199 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.430131    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:21.430360 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.430202    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:23.430182793 +0000 UTC m=+7.719738545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:21.642448 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.642386    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 12:20:19 +0000 UTC" deadline="2028-03-01 06:52:55.705322255 +0000 UTC"
Sep 19 12:25:21.642448 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.642446    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12690h27m34.062880757s"
Sep 19 12:25:21.716604 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.716570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:21.716763 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.716709    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:21.717150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.717127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:21.717262 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.717221    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:21.717608 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.717391    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:21.717608 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:21.717473    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:21.741175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.741139    2568 generic.go:356] "Generic (PLEG): container finished" podID="754257620d708c54c6031bfd4f611893" containerID="65e3cb760ed96e8cb591311fc7f852505d515b02d8b037843f9989e80e4704f8" exitCode=0
Sep 19 12:25:21.741297 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:21.741203    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal" event={"ID":"754257620d708c54c6031bfd4f611893","Type":"ContainerDied","Data":"65e3cb760ed96e8cb591311fc7f852505d515b02d8b037843f9989e80e4704f8"}
Sep 19 12:25:22.755403 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:22.754683    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal" event={"ID":"754257620d708c54c6031bfd4f611893","Type":"ContainerStarted","Data":"e6fc3b40d3fe0bdaf1677e7d31abd7b477a2fa8c30f8251e8cabf4fb776aab64"}
Sep 19 12:25:22.771724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:22.770940    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-161.ec2.internal" podStartSLOduration=4.770921399 podStartE2EDuration="4.770921399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:22.770620112 +0000 UTC m=+7.060175881" watchObservedRunningTime="2026-09-19 12:25:22.770921399 +0000 UTC m=+7.060477171"
Sep 19 12:25:23.249942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.249904    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:23.250116 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.250099    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:23.250188 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.250168    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:27.250149854 +0000 UTC m=+11.539705602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:23.351391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.351355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:23.351590 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.351571    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:23.351659 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.351650    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:27.351631016 +0000 UTC m=+11.641186765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:23.452700 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.452637    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:23.452871 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.452812    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:23.452871 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.452832    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:23.452871 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.452845    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:23.453046 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.452906    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:27.45288799 +0000 UTC m=+11.742443780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:23.716711 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.716679    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:23.716882 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.716846    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:23.717260 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.717238    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:23.717355 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.717333    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:23.717410 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:23.717389    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:23.717468 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:23.717452    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:25.716179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:25.716133    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:25.716179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:25.716159    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:25.716706 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:25.716133    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:25.716706 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:25.716269    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:25.716706 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:25.716358    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:25.716706 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:25.716455    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:27.282077 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.282038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:27.282552 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.282223    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:27.282552 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.282306    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:35.282284436 +0000 UTC m=+19.571840201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:27.383541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.383428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:27.383713 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.383628    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:27.383713 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.383701    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:35.383681957 +0000 UTC m=+19.673237706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:27.484900 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.484804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:27.485100 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.484957    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:27.485100 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.484982    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:27.485100 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.485005    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:27.485100 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.485071    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:35.485052479 +0000 UTC m=+19.774608249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:27.716630 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.716597    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:27.716813 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.716708    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:27.716813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.716761    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:27.716900 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.716841    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:27.717202 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:27.717185    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:27.717310 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:27.717268    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:29.716693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:29.716663    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:29.717220 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:29.716660    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:29.717220 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:29.716797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:29.717220 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:29.716789    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:29.717220 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:29.716899    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:29.717220 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:29.716991    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:31.716545 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:31.716509    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:31.717115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:31.716509    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:31.717115 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:31.716634    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:31.717115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:31.716509    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:31.717115 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:31.716740    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:31.717115 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:31.716809    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:33.716483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:33.716396    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:33.716941 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:33.716403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:33.716941 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:33.716557    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:33.716941 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:33.716605    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:33.716941 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:33.716756    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:33.716941 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:33.716859    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:35.342176 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.342139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:35.342686 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.342309    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:35.342686 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.342394    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:25:51.342372781 +0000 UTC m=+35.631928530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:35.443005 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.442967    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:35.443179 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.443089    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:35.443179 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.443148    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:51.443132493 +0000 UTC m=+35.732688438 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:35.543977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.543931    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:35.544166 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.544114    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:35.544166 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.544139    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:35.544166 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.544149    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:35.544311 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.544212    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:51.544194753 +0000 UTC m=+35.833750504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:35.715968 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.715933    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:35.716162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.715976    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:35.716162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:35.715938    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:35.716162 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.716080    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:35.716313 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.716176    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:35.716313 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:35.716248    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:37.716718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.716565    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:37.716718 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:37.716677    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:37.716718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.716688    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:37.716718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.716708    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:37.717277 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:37.716827    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:37.717277 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:37.716925    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:37.787971 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.787947    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"d5e97bf19776d2c1a1c3b391c971a5233814e63fdd9b475a1089b9f51fa8c7ae"}
Sep 19 12:25:37.789389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.789361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx" event={"ID":"4e665128-4211-4348-a845-ac40c87dd495","Type":"ContainerStarted","Data":"83ca35405ff97b84d8f8fba3ba72a635a1456fa72476e14196772cbb541a9194"}
Sep 19 12:25:37.793923 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.793892    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8v6pn" event={"ID":"b15a0a7b-c722-403f-a655-48d1961671c1","Type":"ContainerStarted","Data":"4489a3117711b6bc08f605efe92df5c20d5576e1b443614ec5ead836f8e71077"}
Sep 19 12:25:37.797756 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.797729    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p2ptk" event={"ID":"bb945c06-675c-4aa3-9c17-37ab108b9b03","Type":"ContainerStarted","Data":"b4d79e8a1687e931172e25c4441e27314eac795c96d106c7fa270b1f63d8c289"}
Sep 19 12:25:37.799100 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.799082    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s7xlq" event={"ID":"12fbbda2-9c99-4465-925a-0a834053447c","Type":"ContainerStarted","Data":"80491cb82e2cd099bc7a9eab341b838d280b1716393be14d2dbd12d0b3ad4ae0"}
Sep 19 12:25:37.800402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.800380    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sbsbv" event={"ID":"aee997a9-8dd4-4ff1-897f-1850eec78e94","Type":"ContainerStarted","Data":"cd91b8a8550859972f92262e2897c24899174d1279070b46f977c5f16f0ef55f"}
Sep 19 12:25:37.801642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.801624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vnjc7" event={"ID":"bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5","Type":"ContainerStarted","Data":"4e4e56d80fa21078a57829c92bea99858c328baeee39997e57bb13a3b11fec65"}
Sep 19 12:25:37.802932 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.802915    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerStarted","Data":"69a5e34d3d991d8d4febf937464979096e2d70314e570db192bc6d6e6149518c"}
Sep 19 12:25:37.807631 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.807592    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8v6pn" podStartSLOduration=3.916160342 podStartE2EDuration="20.807582115s" totalImagesPullingTime="16.891421773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:37.806669176 +0000 UTC m=+22.096224943" watchObservedRunningTime="2026-09-19 12:25:37.807582115 +0000 UTC m=+22.097137883"
Sep 19 12:25:37.823064 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.823027    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p2ptk" podStartSLOduration=3.930720809 podStartE2EDuration="20.823015267s" totalImagesPullingTime="16.892294458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:37.822448408 +0000 UTC m=+22.112004175" watchObservedRunningTime="2026-09-19 12:25:37.823015267 +0000 UTC m=+22.112571035"
Sep 19 12:25:37.835676 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.835620    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sbsbv" podStartSLOduration=7.957280205 podStartE2EDuration="20.83560157s" totalImagesPullingTime="12.878321365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:37.83514639 +0000 UTC m=+22.124702152" watchObservedRunningTime="2026-09-19 12:25:37.83560157 +0000 UTC m=+22.125157400"
Sep 19 12:25:37.868803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.868650    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s7xlq" podStartSLOduration=3.926827722 podStartE2EDuration="20.868636101s" totalImagesPullingTime="16.941808379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:37.868455954 +0000 UTC m=+22.158011722" watchObservedRunningTime="2026-09-19 12:25:37.868636101 +0000 UTC m=+22.158191870"
Sep 19 12:25:37.882353 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.882298    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vnjc7" podStartSLOduration=3.962992576 podStartE2EDuration="20.882281723s" totalImagesPullingTime="16.919289147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:37.88204866 +0000 UTC m=+22.171604429" watchObservedRunningTime="2026-09-19 12:25:37.882281723 +0000 UTC m=+22.171837510"
Sep 19 12:25:37.930602 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:37.930573    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:38.806541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.806514    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="69a5e34d3d991d8d4febf937464979096e2d70314e570db192bc6d6e6149518c" exitCode=0
Sep 19 12:25:38.807176 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.806574    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"69a5e34d3d991d8d4febf937464979096e2d70314e570db192bc6d6e6149518c"}
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.810384    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812477    2568 generic.go:356] "Generic (PLEG): container finished" podID="b122ea00-99d3-47a3-b170-cf888a0ff9af" containerID="9add4381c91033c43d7086a2786fa35dc35f45ff7cc13d16500d3cc2d122e61c" exitCode=1
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812601    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"e15e11b14364746c948614a13f7d7201075c0c163a7815e27c2670c3513bddd8"}
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812734    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"d3b3fc8528a97417032e26c0a52fa6517cdaced8de160e2efe861449c5b9d923"}
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812811    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"5eec6a08de98027a97d5110448d756426ea67803d1b6eeec907d3542a0063fa1"}
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812892    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"9428fab86ad520795639efbce9a46b1d70cf8f5588ae46e53cefe4069b88afd4"}
Sep 19 12:25:38.813152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.812954    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerDied","Data":"9add4381c91033c43d7086a2786fa35dc35f45ff7cc13d16500d3cc2d122e61c"}
Sep 19 12:25:38.815684 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.815619    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbjpc" event={"ID":"e280c2b4-e6d5-4248-b857-9db63cd55f2c","Type":"ContainerStarted","Data":"7298a7b25fd9991380021fc335132edc8c89625dd3d29d9bc663f1eb9e82a697"}
Sep 19 12:25:38.820884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:38.820854    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 12:25:39.685803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.685636    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T12:25:38.820880373Z","UUID":"0c140ce2-bfa4-4b0f-8176-8b18521da77e","Handler":null,"Name":"","Endpoint":""}
Sep 19 12:25:39.686904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.686886    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 12:25:39.687013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.686913    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 12:25:39.715704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.715676    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:39.715886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.715713    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:39.715886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.715676    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:39.715886 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:39.715804    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:39.715886 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:39.715877    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:39.716092 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:39.715945    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:39.821205 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.821165    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx" event={"ID":"4e665128-4211-4348-a845-ac40c87dd495","Type":"ContainerStarted","Data":"098a31e3c794ad39b5eb97313129ecb9046c01cd2aa9ed7abec10d825e741154"}
Sep 19 12:25:39.834470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:39.834427    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nbjpc" podStartSLOduration=5.911908622 podStartE2EDuration="22.834412345s" totalImagesPullingTime="16.922503723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.834225493 +0000 UTC m=+24.123781272" watchObservedRunningTime="2026-09-19 12:25:39.834412345 +0000 UTC m=+24.123968112"
Sep 19 12:25:40.825895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:40.825869    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:25:40.826348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:40.826239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"bce99701dbb83b4b58fc77b68f47ce8723a9f6593d3eeab6659577754dc633fb"}
Sep 19 12:25:40.828134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:40.828106    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx" event={"ID":"4e665128-4211-4348-a845-ac40c87dd495","Type":"ContainerStarted","Data":"572f3af71430064a55e2984a97318490cf2b0a7488b62405e147aae259b59a33"}
Sep 19 12:25:40.845237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:40.845194    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hmjbx" podStartSLOduration=4.763008211 podStartE2EDuration="23.845182328s" totalImagesPullingTime="19.082174117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:40.844561245 +0000 UTC m=+25.134117014" watchObservedRunningTime="2026-09-19 12:25:40.845182328 +0000 UTC m=+25.134738087"
Sep 19 12:25:41.716177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:41.716142    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:41.716355 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:41.716263    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:41.716355 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:41.716306    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:41.716466 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:41.716388    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:41.716466 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:41.716268    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:41.716589 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:41.716536    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:42.594249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:42.594002    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:42.594682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:42.594580    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:42.833119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:42.833053    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sbsbv"
Sep 19 12:25:43.715746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.715723    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:43.716210 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.715724    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:43.716210 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:43.715827    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:43.716210 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:43.715890    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:43.716210 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.715928    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:43.716210 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:43.716017    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:43.837444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.837422    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:25:43.837888 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.837860    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"7e79d5dc45ac32bb5b0edd67fb7ff6a47b9a8d5094014586117426bdb5b961e6"}
Sep 19 12:25:43.838244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.838217    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:43.838336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.838320    2568 scope.go:117] "RemoveContainer" containerID="9add4381c91033c43d7086a2786fa35dc35f45ff7cc13d16500d3cc2d122e61c"
Sep 19 12:25:43.853181 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:43.853160    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:44.841563 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.841520    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="f1bcb19205aa3d72cc3099058e84dce3b246f9cea996a269ce6a5b76ad6e7dba" exitCode=0
Sep 19 12:25:44.842254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.841572    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"f1bcb19205aa3d72cc3099058e84dce3b246f9cea996a269ce6a5b76ad6e7dba"}
Sep 19 12:25:44.845596 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.845576    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:25:44.845928 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.845907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" event={"ID":"b122ea00-99d3-47a3-b170-cf888a0ff9af","Type":"ContainerStarted","Data":"04671319ff7eaa7b9909f2e997a93e23bee888443b1fc366b51955483db4c37a"}
Sep 19 12:25:44.846017 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.846007    2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 19 12:25:44.846230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.846215    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:44.860939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.860918    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:44.900599 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:44.900474    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd" podStartSLOduration=10.995208814 podStartE2EDuration="27.900457427s" totalImagesPullingTime="16.905248613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:44.899905097 +0000 UTC m=+29.189460866" watchObservedRunningTime="2026-09-19 12:25:44.900457427 +0000 UTC m=+29.190013196"
Sep 19 12:25:45.679251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.679056    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-658s9"]
Sep 19 12:25:45.679450 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.679348    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:45.679526 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:45.679454    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:45.685681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.685595    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6cd6x"]
Sep 19 12:25:45.685780 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.685702    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:45.685853 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:45.685833    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:45.690026 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.690001    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l55mc"]
Sep 19 12:25:45.690116 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.690075    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:45.690152 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:45.690140    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:45.848557 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:45.848530    2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 19 12:25:46.031768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:46.031732    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:25:46.852751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:46.852672    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="a54660841d3f3a09214e1ab3b4c15ab5d3dafbfa264a675cd741252625463db9" exitCode=0
Sep 19 12:25:46.853079 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:46.852760    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"a54660841d3f3a09214e1ab3b4c15ab5d3dafbfa264a675cd741252625463db9"}
Sep 19 12:25:47.716262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:47.716237    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:47.716380 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:47.716326    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:47.716445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:47.716427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:47.716581 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:47.716559    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:47.716656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:47.716637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:47.716814 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:47.716716    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:47.857211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:47.857185    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="ad012d44c4aa42cfab2456a0806a7e522c3c2d65ef2fa6270d07839fa2a686e2" exitCode=0
Sep 19 12:25:47.857621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:47.857272    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"ad012d44c4aa42cfab2456a0806a7e522c3c2d65ef2fa6270d07839fa2a686e2"}
Sep 19 12:25:49.715765 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:49.715727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:49.716168 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:49.715727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:49.716168 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:49.715843    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l55mc" podUID="3963b826-8202-4541-accd-847d1da60ae9"
Sep 19 12:25:49.716168 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:49.715727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:49.716168 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:49.715928    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6cd6x" podUID="49b956e3-20ce-47e3-9c91-d7f4194127d3"
Sep 19 12:25:49.716168 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:49.716037    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-658s9" podUID="6d9c3687-5584-4740-8d07-8cc36bb8213b"
Sep 19 12:25:50.725053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.724861    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-161.ec2.internal" event="NodeReady"
Sep 19 12:25:50.725441 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.725155    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 12:25:50.780980 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.780906    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:25:50.807581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.807554    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fkcc7"]
Sep 19 12:25:50.807846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.807695    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.811517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.811477    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 12:25:50.811679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.811662    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 12:25:50.811800 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.811785    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qgdlj\""
Sep 19 12:25:50.812789 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.812761    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 12:25:50.817786 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.817768    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 12:25:50.822150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.822131    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vgq6p"]
Sep 19 12:25:50.822258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.822244    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:50.825848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.825830    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 12:25:50.825944 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.825868    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kdhkj\""
Sep 19 12:25:50.826165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.826149    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 12:25:50.826259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.826242    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 12:25:50.840432 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.840410    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fkcc7"]
Sep 19 12:25:50.840552 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.840452    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:25:50.840552 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.840476    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vgq6p"]
Sep 19 12:25:50.840657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.840578    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:50.843951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.843903    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 12:25:50.843951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.843929    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 12:25:50.844112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.843951    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6jvqq\""
Sep 19 12:25:50.947819 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.947788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.947819 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.947822    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:50.948034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.947851    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.947879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jchqt\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.947998    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948032    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwkj5\" (UniqueName: \"kubernetes.io/projected/a3d03cb6-902e-40e1-8a19-3241c94325ba-kube-api-access-kwkj5\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:50.948165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948061    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c8aff9-8f5e-4542-a402-547ae16192a6-config-volume\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:50.948165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948120    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2c8aff9-8f5e-4542-a402-547ae16192a6-tmp-dir\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:50.948165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nnbb\" (UniqueName: \"kubernetes.io/projected/a2c8aff9-8f5e-4542-a402-547ae16192a6-kube-api-access-2nnbb\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:50.948341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948172    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948243    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:50.948341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:50.948278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d03cb6-902e-40e1-8a19-3241c94325ba-cert\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.049208 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049130    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049208 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049173    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d03cb6-902e-40e1-8a19-3241c94325ba-cert\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.049208 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049206    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049235    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.049468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jchqt\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049468 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.049363    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.049468 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.049437    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls podName:a2c8aff9-8f5e-4542-a402-547ae16192a6 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:51.549415505 +0000 UTC m=+35.838971255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls") pod "dns-default-vgq6p" (UID: "a2c8aff9-8f5e-4542-a402-547ae16192a6") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049590    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049607    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwkj5\" (UniqueName: \"kubernetes.io/projected/a3d03cb6-902e-40e1-8a19-3241c94325ba-kube-api-access-kwkj5\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c8aff9-8f5e-4542-a402-547ae16192a6-config-volume\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049706    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.049778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2c8aff9-8f5e-4542-a402-547ae16192a6-tmp-dir\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nnbb\" (UniqueName: \"kubernetes.io/projected/a2c8aff9-8f5e-4542-a402-547ae16192a6-kube-api-access-2nnbb\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049875    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049912    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.049996    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2c8aff9-8f5e-4542-a402-547ae16192a6-tmp-dir\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.050086    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 12:25:51.050109 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.050100    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c86b487d6-7z7lb: secret "image-registry-tls" not found
Sep 19 12:25:51.050388 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.050152    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls podName:985487cc-b211-4aa0-98b7-1022f4298175 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:51.55013745 +0000 UTC m=+35.839693198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls") pod "image-registry-6c86b487d6-7z7lb" (UID: "985487cc-b211-4aa0-98b7-1022f4298175") : secret "image-registry-tls" not found
Sep 19 12:25:51.050388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.050244    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c8aff9-8f5e-4542-a402-547ae16192a6-config-volume\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.050388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.050307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.050960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.050914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.053822 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.053802    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.053922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.053862    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.053922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.053891    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d03cb6-902e-40e1-8a19-3241c94325ba-cert\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.066188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.066129    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jchqt\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.068390 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.068368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nnbb\" (UniqueName: \"kubernetes.io/projected/a2c8aff9-8f5e-4542-a402-547ae16192a6-kube-api-access-2nnbb\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.068592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.068572    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.077040 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.076973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwkj5\" (UniqueName: \"kubernetes.io/projected/a3d03cb6-902e-40e1-8a19-3241c94325ba-kube-api-access-kwkj5\") pod \"ingress-canary-fkcc7\" (UID: \"a3d03cb6-902e-40e1-8a19-3241c94325ba\") " pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.131904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.131869    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fkcc7"
Sep 19 12:25:51.352966 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.352878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:51.353133 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.353041    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:51.353133 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.353120    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret podName:6d9c3687-5584-4740-8d07-8cc36bb8213b nodeName:}" failed. No retries permitted until 2026-09-19 12:26:23.353099546 +0000 UTC m=+67.642655307 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret") pod "global-pull-secret-syncer-658s9" (UID: "6d9c3687-5584-4740-8d07-8cc36bb8213b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:51.454212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.454180    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:51.454395 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.454352    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:51.454441 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.454413    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs podName:49b956e3-20ce-47e3-9c91-d7f4194127d3 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:23.454398965 +0000 UTC m=+67.743954716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs") pod "network-metrics-daemon-6cd6x" (UID: "49b956e3-20ce-47e3-9c91-d7f4194127d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:51.554680 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.554642    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:51.554873 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.554694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:51.554873 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.554734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:51.554873 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554829    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554872    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554897    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554909    2568 projected.go:196] Error preparing data for projected volume kube-api-access-pqqh2 for pod openshift-network-diagnostics/network-check-target-l55mc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554834    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554908    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls podName:a2c8aff9-8f5e-4542-a402-547ae16192a6 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.554887077 +0000 UTC m=+36.844442827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls") pod "dns-default-vgq6p" (UID: "a2c8aff9-8f5e-4542-a402-547ae16192a6") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554955    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c86b487d6-7z7lb: secret "image-registry-tls" not found
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.554967    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2 podName:3963b826-8202-4541-accd-847d1da60ae9 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:23.554952632 +0000 UTC m=+67.844508386 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqqh2" (UniqueName: "kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2") pod "network-check-target-l55mc" (UID: "3963b826-8202-4541-accd-847d1da60ae9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:51.555016 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:51.555008    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls podName:985487cc-b211-4aa0-98b7-1022f4298175 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.554996265 +0000 UTC m=+36.844552011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls") pod "image-registry-6c86b487d6-7z7lb" (UID: "985487cc-b211-4aa0-98b7-1022f4298175") : secret "image-registry-tls" not found
Sep 19 12:25:51.715840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.715807    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:25:51.716018 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.715807    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:25:51.716239 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.715819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:25:51.719383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.719360    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 12:25:51.719383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.719378    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 12:25:51.719772 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.719752    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4klm\""
Sep 19 12:25:51.719998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.719982    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 12:25:51.720096 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.720078    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g8nvf\""
Sep 19 12:25:51.720386 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:51.720346    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 12:25:52.340550 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.340526    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fkcc7"]
Sep 19 12:25:52.343458 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:25:52.343435    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d03cb6_902e_40e1_8a19_3241c94325ba.slice/crio-8c7c9f6a486f90a0994288fd3a16999ef7f6e8eed5380987dc9d3e14f820a78f WatchSource:0}: Error finding container 8c7c9f6a486f90a0994288fd3a16999ef7f6e8eed5380987dc9d3e14f820a78f: Status 404 returned error can't find the container with id 8c7c9f6a486f90a0994288fd3a16999ef7f6e8eed5380987dc9d3e14f820a78f
Sep 19 12:25:52.562881 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.562701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:52.562881 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.562872    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 12:25:52.563130 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.562894    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c86b487d6-7z7lb: secret "image-registry-tls" not found
Sep 19 12:25:52.563130 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.562909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:52.563130 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.562958    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls podName:985487cc-b211-4aa0-98b7-1022f4298175 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.562935909 +0000 UTC m=+38.852491671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls") pod "image-registry-6c86b487d6-7z7lb" (UID: "985487cc-b211-4aa0-98b7-1022f4298175") : secret "image-registry-tls" not found
Sep 19 12:25:52.563130 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.562992    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:52.563130 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.563031    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls podName:a2c8aff9-8f5e-4542-a402-547ae16192a6 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.563021302 +0000 UTC m=+38.852577048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls") pod "dns-default-vgq6p" (UID: "a2c8aff9-8f5e-4542-a402-547ae16192a6") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:52.628881 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.628840    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-849f4"]
Sep 19 12:25:52.645100 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.645073    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-849f4"]
Sep 19 12:25:52.645240 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.645220    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.649390 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.649367    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-765zc\""
Sep 19 12:25:52.649523 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.649385    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 12:25:52.649523 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.649408    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 12:25:52.650357 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.650339    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 12:25:52.650357 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.650354    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 12:25:52.764734 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.764702    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.764734 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.764735    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.764999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.764819    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e436cfc9-8f66-4fb2-a463-4aa69d714052-crio-socket\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.764999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.764859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e436cfc9-8f66-4fb2-a463-4aa69d714052-data-volume\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.764999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.764883    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px5j5\" (UniqueName: \"kubernetes.io/projected/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-api-access-px5j5\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866129 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866017    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866129 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.866179    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.866193    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866234    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e436cfc9-8f66-4fb2-a463-4aa69d714052-crio-socket\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.866254    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:53.366238452 +0000 UTC m=+37.655794198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:52.866289    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:53.366272152 +0000 UTC m=+37.655827898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:52.866366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866356    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e436cfc9-8f66-4fb2-a463-4aa69d714052-data-volume\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866381    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5j5\" (UniqueName: \"kubernetes.io/projected/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-api-access-px5j5\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866455    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e436cfc9-8f66-4fb2-a463-4aa69d714052-crio-socket\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.866776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.866752    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e436cfc9-8f66-4fb2-a463-4aa69d714052-data-volume\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:52.871693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.871662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fkcc7" event={"ID":"a3d03cb6-902e-40e1-8a19-3241c94325ba","Type":"ContainerStarted","Data":"8c7c9f6a486f90a0994288fd3a16999ef7f6e8eed5380987dc9d3e14f820a78f"}
Sep 19 12:25:52.875098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.875072    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="cefd77a12bc7af0135fd58e1da07a0b64811837d51ce2981ef529f2b8dd6c0da" exitCode=0
Sep 19 12:25:52.875215 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.875105    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"cefd77a12bc7af0135fd58e1da07a0b64811837d51ce2981ef529f2b8dd6c0da"}
Sep 19 12:25:52.889758 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:52.889738    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5j5\" (UniqueName: \"kubernetes.io/projected/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-api-access-px5j5\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:53.370219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:53.370179    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:53.370785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:53.370226    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:53.370785 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:53.370309    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:53.370785 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:53.370391    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.370370929 +0000 UTC m=+38.659926675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:53.370785 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:53.370407    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:53.370785 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:53.370460    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.370442675 +0000 UTC m=+38.659998426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:53.882326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:53.882295    2568 generic.go:356] "Generic (PLEG): container finished" podID="049d77b8-c91f-48ce-badc-f9b904873b7f" containerID="17848c0b3d2a25938022c486fd15653d14b345d8274afa8cdf1ab9c17bcec50b" exitCode=0
Sep 19 12:25:53.882489 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:53.882361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerDied","Data":"17848c0b3d2a25938022c486fd15653d14b345d8274afa8cdf1ab9c17bcec50b"}
Sep 19 12:25:54.378025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.377983    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:54.378458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.378032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:54.378458 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.378133    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:54.378458 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.378207    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:56.378187164 +0000 UTC m=+40.667742912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:54.378458 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.378274    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:54.378458 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.378323    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:56.378311466 +0000 UTC m=+40.667867231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:54.579986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.579954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:54.580167 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.580008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:54.580167 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.580115    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:54.580167 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.580119    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 12:25:54.580167 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.580137    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c86b487d6-7z7lb: secret "image-registry-tls" not found
Sep 19 12:25:54.580338 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.580177    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls podName:a2c8aff9-8f5e-4542-a402-547ae16192a6 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:58.58016095 +0000 UTC m=+42.869716713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls") pod "dns-default-vgq6p" (UID: "a2c8aff9-8f5e-4542-a402-547ae16192a6") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:54.580338 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:54.580191    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls podName:985487cc-b211-4aa0-98b7-1022f4298175 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:58.580185168 +0000 UTC m=+42.869740915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls") pod "image-registry-6c86b487d6-7z7lb" (UID: "985487cc-b211-4aa0-98b7-1022f4298175") : secret "image-registry-tls" not found
Sep 19 12:25:54.582918 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.582893    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8v6pn_b15a0a7b-c722-403f-a655-48d1961671c1/dns-node-resolver/0.log"
Sep 19 12:25:54.888575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.888531    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" event={"ID":"049d77b8-c91f-48ce-badc-f9b904873b7f","Type":"ContainerStarted","Data":"4b4c715d83ef7691f13bb9259146f3ad6c382eff1c826146a50167216b2e0d79"}
Sep 19 12:25:54.890087 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.890064    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fkcc7" event={"ID":"a3d03cb6-902e-40e1-8a19-3241c94325ba","Type":"ContainerStarted","Data":"f00a3f4145fa7eac847a380443591be017d2eb01b8025f415af5c2f0b34dd403"}
Sep 19 12:25:54.909714 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.909670    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qt8cr" podStartSLOduration=9.378818274 podStartE2EDuration="37.909653483s" totalImagesPullingTime="28.530835209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:54.909184621 +0000 UTC m=+39.198740388" watchObservedRunningTime="2026-09-19 12:25:54.909653483 +0000 UTC m=+39.199209246"
Sep 19 12:25:54.928632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:54.928549    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fkcc7" podStartSLOduration=2.6099450429999997 podStartE2EDuration="4.928531602s" totalImagesPullingTime="2.318586559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:54.927787052 +0000 UTC m=+39.217342820" watchObservedRunningTime="2026-09-19 12:25:54.928531602 +0000 UTC m=+39.218087371"
Sep 19 12:25:55.784585 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:55.784401    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vnjc7_bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5/node-ca/0.log"
Sep 19 12:25:56.388590 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:56.388553    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fkcc7_a3d03cb6-902e-40e1-8a19-3241c94325ba/serve-healthcheck-canary/0.log"
Sep 19 12:25:56.394723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:56.394702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:56.394799 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:56.394730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:25:56.394840 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:56.394821    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:56.394905 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:56.394895    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:00.394883021 +0000 UTC m=+44.684438772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:56.394959 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:56.394916    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:56.394995 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:56.394964    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:00.394952378 +0000 UTC m=+44.684508129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:58.610882 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:58.610840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:25:58.610947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:58.610995    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:58.611055    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:58.611066    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c86b487d6-7z7lb: secret "image-registry-tls" not found
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:58.611057    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls podName:a2c8aff9-8f5e-4542-a402-547ae16192a6 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:06.611042768 +0000 UTC m=+50.900598514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls") pod "dns-default-vgq6p" (UID: "a2c8aff9-8f5e-4542-a402-547ae16192a6") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:58.611258 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:25:58.611116    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls podName:985487cc-b211-4aa0-98b7-1022f4298175 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:06.611102498 +0000 UTC m=+50.900658246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls") pod "image-registry-6c86b487d6-7z7lb" (UID: "985487cc-b211-4aa0-98b7-1022f4298175") : secret "image-registry-tls" not found
Sep 19 12:26:00.423681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:00.423612    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:00.423681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:00.423691    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:00.424127 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:26:00.423802    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:26:00.424127 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:26:00.423859    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls podName:e436cfc9-8f66-4fb2-a463-4aa69d714052 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:08.423843646 +0000 UTC m=+52.713399418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls") pod "insights-runtime-extractor-849f4" (UID: "e436cfc9-8f66-4fb2-a463-4aa69d714052") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:26:00.424206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:00.424193    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e436cfc9-8f66-4fb2-a463-4aa69d714052-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:06.676528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.676471    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:26:06.676903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.676590    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:06.679574 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.679548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2c8aff9-8f5e-4542-a402-547ae16192a6-metrics-tls\") pod \"dns-default-vgq6p\" (UID: \"a2c8aff9-8f5e-4542-a402-547ae16192a6\") " pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:26:06.679706 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.679623    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"image-registry-6c86b487d6-7z7lb\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") " pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:06.719295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.719266    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:06.751221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:06.751193    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:26:07.028473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.028391    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:26:07.030961 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:07.030936    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985487cc_b211_4aa0_98b7_1022f4298175.slice/crio-2f62816610b1ff226bcb89e32c04e60cc471e25b5e4eee3b8b40ea57628ec665 WatchSource:0}: Error finding container 2f62816610b1ff226bcb89e32c04e60cc471e25b5e4eee3b8b40ea57628ec665: Status 404 returned error can't find the container with id 2f62816610b1ff226bcb89e32c04e60cc471e25b5e4eee3b8b40ea57628ec665
Sep 19 12:26:07.048370 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.048348    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vgq6p"]
Sep 19 12:26:07.051274 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:07.051251    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c8aff9_8f5e_4542_a402_547ae16192a6.slice/crio-a36eabe394f7e0821c5ac876fe3d11edb561646f2276f2f65938193d1ca7e9b6 WatchSource:0}: Error finding container a36eabe394f7e0821c5ac876fe3d11edb561646f2276f2f65938193d1ca7e9b6: Status 404 returned error can't find the container with id a36eabe394f7e0821c5ac876fe3d11edb561646f2276f2f65938193d1ca7e9b6
Sep 19 12:26:07.922268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.922234    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vgq6p" event={"ID":"a2c8aff9-8f5e-4542-a402-547ae16192a6","Type":"ContainerStarted","Data":"a36eabe394f7e0821c5ac876fe3d11edb561646f2276f2f65938193d1ca7e9b6"}
Sep 19 12:26:07.924006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.923972    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" event={"ID":"985487cc-b211-4aa0-98b7-1022f4298175","Type":"ContainerStarted","Data":"12c1ee5955ab2d4d5465400d8a2f82c409dfdd571685bf1de19ea60e0b5f3324"}
Sep 19 12:26:07.924157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.924028    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" event={"ID":"985487cc-b211-4aa0-98b7-1022f4298175","Type":"ContainerStarted","Data":"2f62816610b1ff226bcb89e32c04e60cc471e25b5e4eee3b8b40ea57628ec665"}
Sep 19 12:26:07.924157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.924108    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:07.951292 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:07.950021    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" podStartSLOduration=25.950001754 podStartE2EDuration="25.950001754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:07.947855927 +0000 UTC m=+52.237411700" watchObservedRunningTime="2026-09-19 12:26:07.950001754 +0000 UTC m=+52.239557522"
Sep 19 12:26:08.490937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:08.490905    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:08.493531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:08.493482    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e436cfc9-8f66-4fb2-a463-4aa69d714052-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849f4\" (UID: \"e436cfc9-8f66-4fb2-a463-4aa69d714052\") " pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:08.554359 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:08.554325    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-849f4"
Sep 19 12:26:09.101835 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.101759    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-849f4"]
Sep 19 12:26:09.105957 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:09.105921    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode436cfc9_8f66_4fb2_a463_4aa69d714052.slice/crio-0ccff29c6781361b31c7704b2603f939c709b8d770412afef7df780ff0f35897 WatchSource:0}: Error finding container 0ccff29c6781361b31c7704b2603f939c709b8d770412afef7df780ff0f35897: Status 404 returned error can't find the container with id 0ccff29c6781361b31c7704b2603f939c709b8d770412afef7df780ff0f35897
Sep 19 12:26:09.931749 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.931712    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vgq6p" event={"ID":"a2c8aff9-8f5e-4542-a402-547ae16192a6","Type":"ContainerStarted","Data":"c72671d59f782b88841ba29904bce08f1ba72c4f5d5e531ad71da6fdbc64127f"}
Sep 19 12:26:09.931954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.931770    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vgq6p" event={"ID":"a2c8aff9-8f5e-4542-a402-547ae16192a6","Type":"ContainerStarted","Data":"7b699a4f908a4a7dad3f83270b7df2cf76ced66b361fa4d751a2c4b2ce41f607"}
Sep 19 12:26:09.931954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.931831    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:26:09.933262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.933243    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849f4" event={"ID":"e436cfc9-8f66-4fb2-a463-4aa69d714052","Type":"ContainerStarted","Data":"07359064177cfadc8e25086c431b55efbf0d099089380eae7f7a20cf757eda72"}
Sep 19 12:26:09.933358 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.933283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849f4" event={"ID":"e436cfc9-8f66-4fb2-a463-4aa69d714052","Type":"ContainerStarted","Data":"0ccff29c6781361b31c7704b2603f939c709b8d770412afef7df780ff0f35897"}
Sep 19 12:26:09.950827 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:09.950785    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vgq6p" podStartSLOduration=17.938864377 podStartE2EDuration="19.950772266s" totalImagesPullingTime="2.011907889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:09.949299561 +0000 UTC m=+54.238855329" watchObservedRunningTime="2026-09-19 12:26:09.950772266 +0000 UTC m=+54.240328033"
Sep 19 12:26:10.941066 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:10.941032    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849f4" event={"ID":"e436cfc9-8f66-4fb2-a463-4aa69d714052","Type":"ContainerStarted","Data":"a7ff53071a00eb30ae7dbb8f63794584b9194ba7fff34327dcd4f69a7b271543"}
Sep 19 12:26:12.949709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:12.949673    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849f4" event={"ID":"e436cfc9-8f66-4fb2-a463-4aa69d714052","Type":"ContainerStarted","Data":"7bba646f34fae9f9a76fbc48db029ba75a81d503a8e7b9cc5a73316903c9d487"}
Sep 19 12:26:12.990603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:12.990554    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-849f4" podStartSLOduration=18.010359898 podStartE2EDuration="20.990540387s" totalImagesPullingTime="2.980180489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:12.988995724 +0000 UTC m=+57.278551485" watchObservedRunningTime="2026-09-19 12:26:12.990540387 +0000 UTC m=+57.280096156"
Sep 19 12:26:16.797025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.796976    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"]
Sep 19 12:26:16.799809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.799783    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"]
Sep 19 12:26:16.801652 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.801615    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.802528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.802479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.804351 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.804331    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 19 12:26:16.805361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.805339    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 12:26:16.805361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.805353    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 12:26:16.805515 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.805353    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cslmj\""
Sep 19 12:26:16.805515 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.805443    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 12:26:16.805621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.805609    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 19 12:26:16.810657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.810635    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"]
Sep 19 12:26:16.811361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.811345    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"]
Sep 19 12:26:16.813701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.813679    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:26:16.847341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.847319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kxf9\" (UniqueName: \"kubernetes.io/projected/cb20c336-5d0b-4d15-9733-d56f0583a7b1-kube-api-access-9kxf9\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.847464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.847366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb20c336-5d0b-4d15-9733-d56f0583a7b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.847464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.847383    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3def3961-77d0-4378-8de3-73425a893b15-tmp\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.847464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.847430    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3def3961-77d0-4378-8de3-73425a893b15-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.847587 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.847464    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdtnn\" (UniqueName: \"kubernetes.io/projected/3def3961-77d0-4378-8de3-73425a893b15-kube-api-access-kdtnn\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.899149 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.899122    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b79bc9699-l2nzx"]
Sep 19 12:26:16.900748 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.900734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.918281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.918259    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b79bc9699-l2nzx"]
Sep 19 12:26:16.948232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3def3961-77d0-4378-8de3-73425a893b15-tmp\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.948374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb20c336-5d0b-4d15-9733-d56f0583a7b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.948374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948264    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-installation-pull-secrets\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948286    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdtnn\" (UniqueName: \"kubernetes.io/projected/3def3961-77d0-4378-8de3-73425a893b15-kube-api-access-kdtnn\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.948374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948304    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12a7200d-b7c5-4b65-9541-1091a186fb31-ca-trust-extracted\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948430    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kxf9\" (UniqueName: \"kubernetes.io/projected/cb20c336-5d0b-4d15-9733-d56f0583a7b1-kube-api-access-9kxf9\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948459    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-certificates\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948519    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3def3961-77d0-4378-8de3-73425a893b15-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948538    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-tls\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg28q\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-kube-api-access-cg28q\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-trusted-ca\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-image-registry-private-configuration\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.948897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948676    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3def3961-77d0-4378-8de3-73425a893b15-tmp\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.948897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.948724    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-bound-sa-token\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:16.950698 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.950678    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb20c336-5d0b-4d15-9733-d56f0583a7b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:16.950856 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.950840    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3def3961-77d0-4378-8de3-73425a893b15-klusterlet-config\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.963226 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.963197    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdtnn\" (UniqueName: \"kubernetes.io/projected/3def3961-77d0-4378-8de3-73425a893b15-kube-api-access-kdtnn\") pod \"klusterlet-addon-workmgr-549bc48c6-6b67p\" (UID: \"3def3961-77d0-4378-8de3-73425a893b15\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:16.963720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:16.963704    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kxf9\" (UniqueName: \"kubernetes.io/projected/cb20c336-5d0b-4d15-9733-d56f0583a7b1-kube-api-access-9kxf9\") pod \"managed-serviceaccount-addon-agent-55b86c7568-vrgjz\" (UID: \"cb20c336-5d0b-4d15-9733-d56f0583a7b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:17.049329 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049258    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-trusted-ca\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049329 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-image-registry-private-configuration\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049406    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-bound-sa-token\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-installation-pull-secrets\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049535    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12a7200d-b7c5-4b65-9541-1091a186fb31-ca-trust-extracted\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049569    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-certificates\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049669 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049616    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-tls\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049669 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049642    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg28q\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-kube-api-access-cg28q\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.049985 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.049953    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12a7200d-b7c5-4b65-9541-1091a186fb31-ca-trust-extracted\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.050372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.050354    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-certificates\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.050372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.050365    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12a7200d-b7c5-4b65-9541-1091a186fb31-trusted-ca\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.052113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.052088    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-registry-tls\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.052207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.052179    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-image-registry-private-configuration\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.052270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.052247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12a7200d-b7c5-4b65-9541-1091a186fb31-installation-pull-secrets\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.059186 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.059169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-bound-sa-token\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.062091 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.062071    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg28q\" (UniqueName: \"kubernetes.io/projected/12a7200d-b7c5-4b65-9541-1091a186fb31-kube-api-access-cg28q\") pod \"image-registry-7b79bc9699-l2nzx\" (UID: \"12a7200d-b7c5-4b65-9541-1091a186fb31\") " pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.124717 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.124686    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"
Sep 19 12:26:17.130447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.130430    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:17.208475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.208402    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.401827 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.401590    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz"]
Sep 19 12:26:17.402479 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.402397    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"]
Sep 19 12:26:17.404910 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:17.404887    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb20c336_5d0b_4d15_9733_d56f0583a7b1.slice/crio-26aad727a3599d95b3bca9f63f7b3baca31878347bd28f121bd7a1f208a220f7 WatchSource:0}: Error finding container 26aad727a3599d95b3bca9f63f7b3baca31878347bd28f121bd7a1f208a220f7: Status 404 returned error can't find the container with id 26aad727a3599d95b3bca9f63f7b3baca31878347bd28f121bd7a1f208a220f7
Sep 19 12:26:17.405409 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:17.405388    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3def3961_77d0_4378_8de3_73425a893b15.slice/crio-f1a5e93186a83cce942f28b39b0b35d43bc723ca2093091b6c35e0cb1452a63f WatchSource:0}: Error finding container f1a5e93186a83cce942f28b39b0b35d43bc723ca2093091b6c35e0cb1452a63f: Status 404 returned error can't find the container with id f1a5e93186a83cce942f28b39b0b35d43bc723ca2093091b6c35e0cb1452a63f
Sep 19 12:26:17.491620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.491580    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b79bc9699-l2nzx"]
Sep 19 12:26:17.494733 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:17.494706    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a7200d_b7c5_4b65_9541_1091a186fb31.slice/crio-a077fac859d79618e71701b21efe5c27994d67088c600fc4bcc68c5102833563 WatchSource:0}: Error finding container a077fac859d79618e71701b21efe5c27994d67088c600fc4bcc68c5102833563: Status 404 returned error can't find the container with id a077fac859d79618e71701b21efe5c27994d67088c600fc4bcc68c5102833563
Sep 19 12:26:17.868082 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.868006    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhggd"
Sep 19 12:26:17.965647 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.965612    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz" event={"ID":"cb20c336-5d0b-4d15-9733-d56f0583a7b1","Type":"ContainerStarted","Data":"26aad727a3599d95b3bca9f63f7b3baca31878347bd28f121bd7a1f208a220f7"}
Sep 19 12:26:17.967080 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.967051    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx" event={"ID":"12a7200d-b7c5-4b65-9541-1091a186fb31","Type":"ContainerStarted","Data":"4e63a6fbbc49fb50856ab898a47e93deb505b5c2a18399198548a329909b3d1d"}
Sep 19 12:26:17.967224 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.967104    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx" event={"ID":"12a7200d-b7c5-4b65-9541-1091a186fb31","Type":"ContainerStarted","Data":"a077fac859d79618e71701b21efe5c27994d67088c600fc4bcc68c5102833563"}
Sep 19 12:26:17.967224 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.967133    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:17.968131 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.968109    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p" event={"ID":"3def3961-77d0-4378-8de3-73425a893b15","Type":"ContainerStarted","Data":"f1a5e93186a83cce942f28b39b0b35d43bc723ca2093091b6c35e0cb1452a63f"}
Sep 19 12:26:17.988530 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:17.988477    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx" podStartSLOduration=1.9884630589999999 podStartE2EDuration="1.988463059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:17.987981043 +0000 UTC m=+62.277536811" watchObservedRunningTime="2026-09-19 12:26:17.988463059 +0000 UTC m=+62.278018828"
Sep 19 12:26:19.944261 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:19.943533    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vgq6p"
Sep 19 12:26:21.992574 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:21.992534    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz" event={"ID":"cb20c336-5d0b-4d15-9733-d56f0583a7b1","Type":"ContainerStarted","Data":"8982b6567c3b4b900e1298cca0c705b82599b7e0d0788af37f7cfc188665c979"}
Sep 19 12:26:21.994512 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:21.994467    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p" event={"ID":"3def3961-77d0-4378-8de3-73425a893b15","Type":"ContainerStarted","Data":"ba0f835a0901610e382df47d1ec37d1c2b55990e928443188a0af47096684b3f"}
Sep 19 12:26:21.994703 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:21.994679    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:21.996294 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:21.996257    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p" podUID="3def3961-77d0-4378-8de3-73425a893b15" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.11:8000/readyz\": dial tcp 10.133.0.11:8000: connect: connection refused"
Sep 19 12:26:22.008078 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:22.008026    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55b86c7568-vrgjz" podStartSLOduration=1.520927886 podStartE2EDuration="6.008006132s" totalImagesPullingTime="4.487078246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:22.006641929 +0000 UTC m=+66.296197697" watchObservedRunningTime="2026-09-19 12:26:22.008006132 +0000 UTC m=+66.297561902"
Sep 19 12:26:22.024564 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:22.024491    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p" podStartSLOduration=1.515045208 podStartE2EDuration="6.024469702s" totalImagesPullingTime="4.509424494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:22.024216388 +0000 UTC m=+66.313772158" watchObservedRunningTime="2026-09-19 12:26:22.024469702 +0000 UTC m=+66.314025472"
Sep 19 12:26:22.998138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:22.998111    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-549bc48c6-6b67p"
Sep 19 12:26:23.407345 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.407305    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:26:23.410151 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.410128    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 12:26:23.420370 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.420337    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d9c3687-5584-4740-8d07-8cc36bb8213b-original-pull-secret\") pod \"global-pull-secret-syncer-658s9\" (UID: \"6d9c3687-5584-4740-8d07-8cc36bb8213b\") " pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:26:23.507830 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.507780    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:26:23.511662 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.511638    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 12:26:23.520708 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.520688    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49b956e3-20ce-47e3-9c91-d7f4194127d3-metrics-certs\") pod \"network-metrics-daemon-6cd6x\" (UID: \"49b956e3-20ce-47e3-9c91-d7f4194127d3\") " pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:26:23.527653 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.527635    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-658s9"
Sep 19 12:26:23.541747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.541721    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g8nvf\""
Sep 19 12:26:23.549901 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.549878    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6cd6x"
Sep 19 12:26:23.609176 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.609099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:26:23.612317 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.612293    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 12:26:23.621889 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.621688    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 12:26:23.632109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.632086    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqh2\" (UniqueName: \"kubernetes.io/projected/3963b826-8202-4541-accd-847d1da60ae9-kube-api-access-pqqh2\") pod \"network-check-target-l55mc\" (UID: \"3963b826-8202-4541-accd-847d1da60ae9\") " pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:26:23.798423 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.798356    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-658s9"]
Sep 19 12:26:23.801146 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:23.801120    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9c3687_5584_4740_8d07_8cc36bb8213b.slice/crio-a3cf8f6445ba1eca5acc84aac9c4f02e170a5a670853a6a231d865af9828bbeb WatchSource:0}: Error finding container a3cf8f6445ba1eca5acc84aac9c4f02e170a5a670853a6a231d865af9828bbeb: Status 404 returned error can't find the container with id a3cf8f6445ba1eca5acc84aac9c4f02e170a5a670853a6a231d865af9828bbeb
Sep 19 12:26:23.825246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.825218    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6cd6x"]
Sep 19 12:26:23.827666 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:23.827642    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b956e3_20ce_47e3_9c91_d7f4194127d3.slice/crio-8482f8a12fc22121ba4b5d4d9351e386e178b2110258ffe93c6e4103929b8876 WatchSource:0}: Error finding container 8482f8a12fc22121ba4b5d4d9351e386e178b2110258ffe93c6e4103929b8876: Status 404 returned error can't find the container with id 8482f8a12fc22121ba4b5d4d9351e386e178b2110258ffe93c6e4103929b8876
Sep 19 12:26:23.835992 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.835973    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4klm\""
Sep 19 12:26:23.843920 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:23.843900    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:26:24.000954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:24.000918    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-658s9" event={"ID":"6d9c3687-5584-4740-8d07-8cc36bb8213b","Type":"ContainerStarted","Data":"a3cf8f6445ba1eca5acc84aac9c4f02e170a5a670853a6a231d865af9828bbeb"}
Sep 19 12:26:24.002057 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:24.002038    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6cd6x" event={"ID":"49b956e3-20ce-47e3-9c91-d7f4194127d3","Type":"ContainerStarted","Data":"8482f8a12fc22121ba4b5d4d9351e386e178b2110258ffe93c6e4103929b8876"}
Sep 19 12:26:24.110945 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:24.110874    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l55mc"]
Sep 19 12:26:24.114050 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:24.114023    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3963b826_8202_4541_accd_847d1da60ae9.slice/crio-dfc13b4375a45c735d9907296f4bed8c52e26b5f408b47e483e4aca36bdd1168 WatchSource:0}: Error finding container dfc13b4375a45c735d9907296f4bed8c52e26b5f408b47e483e4aca36bdd1168: Status 404 returned error can't find the container with id dfc13b4375a45c735d9907296f4bed8c52e26b5f408b47e483e4aca36bdd1168
Sep 19 12:26:25.008121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.008089    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6cd6x" event={"ID":"49b956e3-20ce-47e3-9c91-d7f4194127d3","Type":"ContainerStarted","Data":"84a24295ceda4aeb7bb37b685a9ce051238cbcfa22d95dfa8b3991fef3e04ce3"}
Sep 19 12:26:25.011332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.011300    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l55mc" event={"ID":"3963b826-8202-4541-accd-847d1da60ae9","Type":"ContainerStarted","Data":"dfc13b4375a45c735d9907296f4bed8c52e26b5f408b47e483e4aca36bdd1168"}
Sep 19 12:26:25.977868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.977832    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-gthsf"]
Sep 19 12:26:25.979976 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.979954    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:25.983177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.983002    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-s7zmv\""
Sep 19 12:26:25.983177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.983106    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 12:26:25.983177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.983143    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 12:26:25.990812 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:25.990317    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-gthsf"]
Sep 19 12:26:26.019089 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.019046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6cd6x" event={"ID":"49b956e3-20ce-47e3-9c91-d7f4194127d3","Type":"ContainerStarted","Data":"1cb7f9dcc1fd89a10c4db2abd34adf77bdd0fe7f8f7d370f735c85d543013135"}
Sep 19 12:26:26.027722 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.027696    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwp78\" (UniqueName: \"kubernetes.io/projected/e76d3cc9-ef1c-4058-a1fd-7f8684a9792f-kube-api-access-dwp78\") pod \"downloads-6594f4c75-gthsf\" (UID: \"e76d3cc9-ef1c-4058-a1fd-7f8684a9792f\") " pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:26.128775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.128732    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwp78\" (UniqueName: \"kubernetes.io/projected/e76d3cc9-ef1c-4058-a1fd-7f8684a9792f-kube-api-access-dwp78\") pod \"downloads-6594f4c75-gthsf\" (UID: \"e76d3cc9-ef1c-4058-a1fd-7f8684a9792f\") " pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:26.137866 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.137836    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwp78\" (UniqueName: \"kubernetes.io/projected/e76d3cc9-ef1c-4058-a1fd-7f8684a9792f-kube-api-access-dwp78\") pod \"downloads-6594f4c75-gthsf\" (UID: \"e76d3cc9-ef1c-4058-a1fd-7f8684a9792f\") " pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:26.291978 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.291882    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:26.571473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.571359    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6cd6x" podStartSLOduration=68.523532142 podStartE2EDuration="1m9.571334976s" totalImagesPullingTime="1.047802834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:26.040173577 +0000 UTC m=+70.329729348" watchObservedRunningTime="2026-09-19 12:26:26.571334976 +0000 UTC m=+70.860890746"
Sep 19 12:26:26.572072 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.571941    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-gthsf"]
Sep 19 12:26:26.820111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.820077    2568 patch_prober.go:28] interesting pod/image-registry-6c86b487d6-7z7lb container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 19 12:26:26.820288 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:26.820136    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" podUID="985487cc-b211-4aa0-98b7-1022f4298175" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:26:27.327992 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:27.327947    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode76d3cc9_ef1c_4058_a1fd_7f8684a9792f.slice/crio-ccd24581ef36f0cdeb2920fa4b38c8961951a7df510d2b7345e6dfea85d13b0c WatchSource:0}: Error finding container ccd24581ef36f0cdeb2920fa4b38c8961951a7df510d2b7345e6dfea85d13b0c: Status 404 returned error can't find the container with id ccd24581ef36f0cdeb2920fa4b38c8961951a7df510d2b7345e6dfea85d13b0c
Sep 19 12:26:27.954830 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.954796    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"]
Sep 19 12:26:27.956981 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.956961    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:27.959279 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.959251    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 19 12:26:27.960547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.960528    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 12:26:27.960547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.960537    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 12:26:27.960701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.960570    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 12:26:27.960701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.960574    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tljt5\""
Sep 19 12:26:27.960701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.960536    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 19 12:26:27.980866 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.980839    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"]
Sep 19 12:26:27.999551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:27.999525    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6sl6r"]
Sep 19 12:26:28.001929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.001909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.004891 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.004857    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 12:26:28.005038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.004897    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f82qw\""
Sep 19 12:26:28.005038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.004900    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 12:26:28.005180 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.005081    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 12:26:28.029685 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.029654    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-gthsf" event={"ID":"e76d3cc9-ef1c-4058-a1fd-7f8684a9792f","Type":"ContainerStarted","Data":"ccd24581ef36f0cdeb2920fa4b38c8961951a7df510d2b7345e6dfea85d13b0c"}
Sep 19 12:26:28.043013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.042985    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nxmn\" (UniqueName: \"kubernetes.io/projected/103274cb-e53d-4cc1-b242-036ae4a72dfb-kube-api-access-9nxmn\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-wtmp\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043087    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9f5\" (UniqueName: \"kubernetes.io/projected/265f9c7b-2277-4810-a78f-6ac4900c0ec1-kube-api-access-sn9f5\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.043138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043129    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.043304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.043304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-metrics-client-ca\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-textfile\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043298    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043476 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-root\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043476 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043353    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-tls\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.043476 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043378    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265f9c7b-2277-4810-a78f-6ac4900c0ec1-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.043476 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.043410    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-sys\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.143957    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.144002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-metrics-client-ca\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-textfile\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-root\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-tls\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144147    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265f9c7b-2277-4810-a78f-6ac4900c0ec1-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144181    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-sys\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144222    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nxmn\" (UniqueName: \"kubernetes.io/projected/103274cb-e53d-4cc1-b242-036ae4a72dfb-kube-api-access-9nxmn\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144246    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-wtmp\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144270    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144296    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9f5\" (UniqueName: \"kubernetes.io/projected/265f9c7b-2277-4810-a78f-6ac4900c0ec1-kube-api-access-sn9f5\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144327    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144425    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-textfile\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144475    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-sys\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144605    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-wtmp\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144902 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144739    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-metrics-client-ca\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.144972 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.144950    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/103274cb-e53d-4cc1-b242-036ae4a72dfb-root\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.145457 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.145402    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.145702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.145677    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265f9c7b-2277-4810-a78f-6ac4900c0ec1-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.146914 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.146888    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.147098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.147071    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.147209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.147127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265f9c7b-2277-4810-a78f-6ac4900c0ec1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.147282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.147263    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/103274cb-e53d-4cc1-b242-036ae4a72dfb-node-exporter-tls\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.153950 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.153925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9f5\" (UniqueName: \"kubernetes.io/projected/265f9c7b-2277-4810-a78f-6ac4900c0ec1-kube-api-access-sn9f5\") pod \"openshift-state-metrics-86c5bc79d6-pghsz\" (UID: \"265f9c7b-2277-4810-a78f-6ac4900c0ec1\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.154455 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.154429    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nxmn\" (UniqueName: \"kubernetes.io/projected/103274cb-e53d-4cc1-b242-036ae4a72dfb-kube-api-access-9nxmn\") pod \"node-exporter-6sl6r\" (UID: \"103274cb-e53d-4cc1-b242-036ae4a72dfb\") " pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.267691 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.267602    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"
Sep 19 12:26:28.312234 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:28.312206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sl6r"
Sep 19 12:26:28.752959 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:28.752782    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103274cb_e53d_4cc1_b242_036ae4a72dfb.slice/crio-f154714d8824e19392ea73d1e90d6f609192c3b1255704c84d9f8623a37803eb WatchSource:0}: Error finding container f154714d8824e19392ea73d1e90d6f609192c3b1255704c84d9f8623a37803eb: Status 404 returned error can't find the container with id f154714d8824e19392ea73d1e90d6f609192c3b1255704c84d9f8623a37803eb
Sep 19 12:26:29.023683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.023599    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz"]
Sep 19 12:26:29.027437 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:29.027311    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265f9c7b_2277_4810_a78f_6ac4900c0ec1.slice/crio-9dee29403c19a1d561def0c617bb44cd0975140690fc914bfaa2aa52452cb2a4 WatchSource:0}: Error finding container 9dee29403c19a1d561def0c617bb44cd0975140690fc914bfaa2aa52452cb2a4: Status 404 returned error can't find the container with id 9dee29403c19a1d561def0c617bb44cd0975140690fc914bfaa2aa52452cb2a4
Sep 19 12:26:29.035429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.035403    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l55mc" event={"ID":"3963b826-8202-4541-accd-847d1da60ae9","Type":"ContainerStarted","Data":"8a9f9031bbcee444b18c7884a911c3b5926c655c2a241555376181dd6adc8a96"}
Sep 19 12:26:29.035996 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.035790    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:26:29.037363 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.037342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz" event={"ID":"265f9c7b-2277-4810-a78f-6ac4900c0ec1","Type":"ContainerStarted","Data":"9dee29403c19a1d561def0c617bb44cd0975140690fc914bfaa2aa52452cb2a4"}
Sep 19 12:26:29.039053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.039025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sl6r" event={"ID":"103274cb-e53d-4cc1-b242-036ae4a72dfb","Type":"ContainerStarted","Data":"f154714d8824e19392ea73d1e90d6f609192c3b1255704c84d9f8623a37803eb"}
Sep 19 12:26:29.045965 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.045935    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-658s9" event={"ID":"6d9c3687-5584-4740-8d07-8cc36bb8213b","Type":"ContainerStarted","Data":"dcdd17e2df2b8c9666538e5d61ce3ae2649789e1d6637a77da0d24ba274d2dbc"}
Sep 19 12:26:29.062397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.062349    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l55mc" podStartSLOduration=67.421663978 podStartE2EDuration="1m12.062328875s" totalImagesPullingTime="4.640664897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:29.059577355 +0000 UTC m=+73.349133124" watchObservedRunningTime="2026-09-19 12:26:29.062328875 +0000 UTC m=+73.351884648"
Sep 19 12:26:29.085313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:29.085272    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-658s9" podStartSLOduration=67.118045563 podStartE2EDuration="1m12.085255972s" totalImagesPullingTime="4.967210409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:29.084030219 +0000 UTC m=+73.373585987" watchObservedRunningTime="2026-09-19 12:26:29.085255972 +0000 UTC m=+73.374811741"
Sep 19 12:26:30.053844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:30.053697    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz" event={"ID":"265f9c7b-2277-4810-a78f-6ac4900c0ec1","Type":"ContainerStarted","Data":"57594bbd0b0c700c8d3109247e6268fbeadc22d04223b6d329ea508942a7eb86"}
Sep 19 12:26:30.053844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:30.053764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz" event={"ID":"265f9c7b-2277-4810-a78f-6ac4900c0ec1","Type":"ContainerStarted","Data":"a30602508ed830bcfa036384453c25b54655f201082d6642d187cfc5deb5e71b"}
Sep 19 12:26:30.055896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:30.055868    2568 generic.go:356] "Generic (PLEG): container finished" podID="103274cb-e53d-4cc1-b242-036ae4a72dfb" containerID="3161b70248d52344221c50823d37747c6b800079d8e72e65440a6d1409abfa02" exitCode=0
Sep 19 12:26:30.056018 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:30.055956    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sl6r" event={"ID":"103274cb-e53d-4cc1-b242-036ae4a72dfb","Type":"ContainerDied","Data":"3161b70248d52344221c50823d37747c6b800079d8e72e65440a6d1409abfa02"}
Sep 19 12:26:31.062976 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:31.062891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz" event={"ID":"265f9c7b-2277-4810-a78f-6ac4900c0ec1","Type":"ContainerStarted","Data":"3ed5bf89a4e87bdda99f3ce2a503b95542ee5c7024caa5bb4baf24446587950d"}
Sep 19 12:26:31.065844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:31.065816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sl6r" event={"ID":"103274cb-e53d-4cc1-b242-036ae4a72dfb","Type":"ContainerStarted","Data":"71d4b24633c1be24eb2f7d430f754c440da0b4ed4a695fbaf3e30e7b60f8f26b"}
Sep 19 12:26:31.065982 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:31.065863    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sl6r" event={"ID":"103274cb-e53d-4cc1-b242-036ae4a72dfb","Type":"ContainerStarted","Data":"5d086b46b8b46ad08d301de920b060cc6b78ddb956a6751e8689ea0dd2821d45"}
Sep 19 12:26:31.084564 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:31.084490    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pghsz" podStartSLOduration=3.033594587 podStartE2EDuration="4.084474733s" totalImagesPullingTime="1.050880146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:31.083720725 +0000 UTC m=+75.373276492" watchObservedRunningTime="2026-09-19 12:26:31.084474733 +0000 UTC m=+75.374030848"
Sep 19 12:26:31.111471 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:31.111406    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6sl6r" podStartSLOduration=3.201367202 podStartE2EDuration="4.111380995s" totalImagesPullingTime="833.042431ms" totalInitContainerRuntime="76.971362ms" isStatefulPod=true podCreationTimestamp="2026-09-19 12:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:31.105787587 +0000 UTC m=+75.395343358" watchObservedRunningTime="2026-09-19 12:26:31.111380995 +0000 UTC m=+75.400936765"
Sep 19 12:26:32.728190 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.728156    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"]
Sep 19 12:26:32.731053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.731031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.734856 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.734833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 19 12:26:32.734959 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.734906    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qtqws\""
Sep 19 12:26:32.741288 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.741265    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"]
Sep 19 12:26:32.784610 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.784575    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl654\" (UniqueName: \"kubernetes.io/projected/96a74b53-50b6-45ce-8568-3af5f7d227a8-kube-api-access-gl654\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.784785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.784622    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96a74b53-50b6-45ce-8568-3af5f7d227a8-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.885862 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.885821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl654\" (UniqueName: \"kubernetes.io/projected/96a74b53-50b6-45ce-8568-3af5f7d227a8-kube-api-access-gl654\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.886055 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.885885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96a74b53-50b6-45ce-8568-3af5f7d227a8-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.888430 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.888406    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96a74b53-50b6-45ce-8568-3af5f7d227a8-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:32.895092 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:32.895065    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl654\" (UniqueName: \"kubernetes.io/projected/96a74b53-50b6-45ce-8568-3af5f7d227a8-kube-api-access-gl654\") pod \"monitoring-plugin-5984459ff8-77nr2\" (UID: \"96a74b53-50b6-45ce-8568-3af5f7d227a8\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:33.041994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:33.041915    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:33.320328 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:33.320242    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"]
Sep 19 12:26:33.323539 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:33.323488    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a74b53_50b6_45ce_8568_3af5f7d227a8.slice/crio-f962ec0902a59374c81db6b6b41fcaa89f8990b1cc10ca61a6b07734e08eb9e5 WatchSource:0}: Error finding container f962ec0902a59374c81db6b6b41fcaa89f8990b1cc10ca61a6b07734e08eb9e5: Status 404 returned error can't find the container with id f962ec0902a59374c81db6b6b41fcaa89f8990b1cc10ca61a6b07734e08eb9e5
Sep 19 12:26:34.080537 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.080132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2" event={"ID":"96a74b53-50b6-45ce-8568-3af5f7d227a8","Type":"ContainerStarted","Data":"f962ec0902a59374c81db6b6b41fcaa89f8990b1cc10ca61a6b07734e08eb9e5"}
Sep 19 12:26:34.214527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.213441    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:26:34.217336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.217310    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.220746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.220556    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 19 12:26:34.220967 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.220953    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 19 12:26:34.221636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221198    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 12:26:34.221636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221255    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 19 12:26:34.221636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221346    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 19 12:26:34.221636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221201    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wfvb9\""
Sep 19 12:26:34.221636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221450    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 19 12:26:34.222002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221647    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 19 12:26:34.222002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221722    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 19 12:26:34.222002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221739    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-j3ju59n0r68f\""
Sep 19 12:26:34.222002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221889    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 19 12:26:34.222002 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.221936    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 19 12:26:34.222232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.222092    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 19 12:26:34.222232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.222136    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 19 12:26:34.222232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.222092    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 19 12:26:34.222624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.222605    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 19 12:26:34.224475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.224451    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 19 12:26:34.236582 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.236552    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:26:34.297762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297727    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.297930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.297930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.297930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297836    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.297930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.297930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297889    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.297958    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cflbc\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298040    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298093    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298372    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.298478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.298414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.398986    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399065    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399114    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399252    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399312    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399335    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399414    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399515    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399570    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.399728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399644    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cflbc\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.400217 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.399918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.400714 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.400370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.403759 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.403484    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.404821 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.404589    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.405029 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.405000    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.405118 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.405063    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.405315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.405293    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.405399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.405386    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.406140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.406114    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.408142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.408099    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.409336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.409238    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.409336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.409269    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cflbc\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.409336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.409298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.410185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.410161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.411253 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.411228    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.411651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.411611    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.411746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.411725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.412222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.412185    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.412377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.412348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.413648 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.413623    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.534077 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.534035    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:26:34.946225 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:34.946144    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:26:34.949475 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:26:34.949446    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f21207_8b46_4d1c_8fdb_64752d2779eb.slice/crio-bf4e51763acd5921ef33954413a183459d8b8a765b03aedc7ba7b3f595805d9c WatchSource:0}: Error finding container bf4e51763acd5921ef33954413a183459d8b8a765b03aedc7ba7b3f595805d9c: Status 404 returned error can't find the container with id bf4e51763acd5921ef33954413a183459d8b8a765b03aedc7ba7b3f595805d9c
Sep 19 12:26:35.086866 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:35.086038    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"bf4e51763acd5921ef33954413a183459d8b8a765b03aedc7ba7b3f595805d9c"}
Sep 19 12:26:35.088742 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:35.088714    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2" event={"ID":"96a74b53-50b6-45ce-8568-3af5f7d227a8","Type":"ContainerStarted","Data":"afc122c7a89ec51c4a1bf838caf21c4d34169bfb56fb5825c6c1c59baac1bb8d"}
Sep 19 12:26:35.088968 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:35.088919    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:35.094689 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:35.094664    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2"
Sep 19 12:26:35.105822 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:35.105725    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-77nr2" podStartSLOduration=1.7201728649999999 podStartE2EDuration="3.105705877s" totalImagesPullingTime="1.385533012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:35.105059724 +0000 UTC m=+79.394615492" watchObservedRunningTime="2026-09-19 12:26:35.105705877 +0000 UTC m=+79.395261652"
Sep 19 12:26:36.095054 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:36.095017    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3" exitCode=0
Sep 19 12:26:36.095538 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:36.095133    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3"}
Sep 19 12:26:36.820058 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:36.820024    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:38.981402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:38.981370    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b79bc9699-l2nzx"
Sep 19 12:26:41.833489 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:41.833435    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" podUID="985487cc-b211-4aa0-98b7-1022f4298175" containerName="registry" containerID="cri-o://12c1ee5955ab2d4d5465400d8a2f82c409dfdd571685bf1de19ea60e0b5f3324" gracePeriod=30
Sep 19 12:26:45.134299 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:45.134240    2568 generic.go:356] "Generic (PLEG): container finished" podID="985487cc-b211-4aa0-98b7-1022f4298175" containerID="12c1ee5955ab2d4d5465400d8a2f82c409dfdd571685bf1de19ea60e0b5f3324" exitCode=0
Sep 19 12:26:45.134299 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:45.134287    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" event={"ID":"985487cc-b211-4aa0-98b7-1022f4298175","Type":"ContainerDied","Data":"12c1ee5955ab2d4d5465400d8a2f82c409dfdd571685bf1de19ea60e0b5f3324"}
Sep 19 12:26:45.937207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:45.937180    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:46.010285 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010253    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010294    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010314    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010377    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jchqt\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010409    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010554    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010612    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010652    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") pod \"985487cc-b211-4aa0-98b7-1022f4298175\" (UID: \"985487cc-b211-4aa0-98b7-1022f4298175\") "
Sep 19 12:26:46.010928 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010822    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:26:46.010928 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010830    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:26:46.011033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010947    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-registry-certificates\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.011033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.010971    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/985487cc-b211-4aa0-98b7-1022f4298175-trusted-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.013012 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.012982    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:26:46.013124 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.013026    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt" (OuterVolumeSpecName: "kube-api-access-jchqt") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "kube-api-access-jchqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:26:46.013124 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.013108    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:26:46.013207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.013132    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:26:46.013207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.013178    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:26:46.019391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.019364    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "985487cc-b211-4aa0-98b7-1022f4298175" (UID: "985487cc-b211-4aa0-98b7-1022f4298175"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:26:46.111403 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111378    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jchqt\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-kube-api-access-jchqt\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.111483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111408    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/985487cc-b211-4aa0-98b7-1022f4298175-ca-trust-extracted\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.111483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111425    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-image-registry-private-configuration\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.111483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111435    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-registry-tls\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.111483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111449    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/985487cc-b211-4aa0-98b7-1022f4298175-bound-sa-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.111483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.111462    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/985487cc-b211-4aa0-98b7-1022f4298175-installation-pull-secrets\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:26:46.140313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.140279    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb" event={"ID":"985487cc-b211-4aa0-98b7-1022f4298175","Type":"ContainerDied","Data":"2f62816610b1ff226bcb89e32c04e60cc471e25b5e4eee3b8b40ea57628ec665"}
Sep 19 12:26:46.140814 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.140369    2568 scope.go:117] "RemoveContainer" containerID="12c1ee5955ab2d4d5465400d8a2f82c409dfdd571685bf1de19ea60e0b5f3324"
Sep 19 12:26:46.140814 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.140300    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c86b487d6-7z7lb"
Sep 19 12:26:46.168071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.168019    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:26:46.177315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:46.177288    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c86b487d6-7z7lb"]
Sep 19 12:26:47.150786 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.150747    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"}
Sep 19 12:26:47.151298 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.150836    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"}
Sep 19 12:26:47.153144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.153066    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-gthsf" event={"ID":"e76d3cc9-ef1c-4058-a1fd-7f8684a9792f","Type":"ContainerStarted","Data":"2203a0de98cf76545ff896432936bab9bc2915b90c3722ccd318bbafd5032501"}
Sep 19 12:26:47.153468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.153427    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:47.165962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.165929    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-gthsf"
Sep 19 12:26:47.173177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.173120    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-gthsf" podStartSLOduration=3.392750283 podStartE2EDuration="22.17309836s" totalImagesPullingTime="18.780348077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:47.172133936 +0000 UTC m=+91.461689707" watchObservedRunningTime="2026-09-19 12:26:47.17309836 +0000 UTC m=+91.462654131"
Sep 19 12:26:47.722100 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:47.722071    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985487cc-b211-4aa0-98b7-1022f4298175" path="/var/lib/kubelet/pods/985487cc-b211-4aa0-98b7-1022f4298175/volumes"
Sep 19 12:26:50.171765 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:50.171725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"}
Sep 19 12:26:50.172214 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:50.171800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"}
Sep 19 12:26:50.172214 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:50.171838    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"}
Sep 19 12:26:50.172214 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:50.171870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerStarted","Data":"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"}
Sep 19 12:26:50.208397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:50.208342    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.263630956 podStartE2EDuration="16.208318125s" totalImagesPullingTime="13.925036788s" totalInitContainerRuntime="19.650381ms" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:50.206622015 +0000 UTC m=+94.496177784" watchObservedRunningTime="2026-09-19 12:26:50.208318125 +0000 UTC m=+94.497873894"
Sep 19 12:26:54.534703 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:26:54.534659    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:00.058716 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:00.058681    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l55mc"
Sep 19 12:27:34.535184 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:34.535141    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:34.607156 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:34.607127    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:35.350050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:35.350023    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:52.565580 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.565547    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.565934    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="prometheus" containerID="cri-o://d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef" gracePeriod=600
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.565986    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85" gracePeriod=600
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.566006    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-web" containerID="cri-o://fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225" gracePeriod=600
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.566029    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="thanos-sidecar" containerID="cri-o://dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f" gracePeriod=600
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.566054    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="config-reloader" containerID="cri-o://f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db" gracePeriod=600
Sep 19 12:27:52.566196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:52.566128    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy" containerID="cri-o://3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692" gracePeriod=600
Sep 19 12:27:53.402886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402855    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85" exitCode=0
Sep 19 12:27:53.402886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402879    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692" exitCode=0
Sep 19 12:27:53.402886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402885    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f" exitCode=0
Sep 19 12:27:53.402886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402892    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db" exitCode=0
Sep 19 12:27:53.402886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402898    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef" exitCode=0
Sep 19 12:27:53.403262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.402930    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"}
Sep 19 12:27:53.403262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.403010    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"}
Sep 19 12:27:53.403262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.403061    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"}
Sep 19 12:27:53.403262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.403096    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"}
Sep 19 12:27:53.403262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.403143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"}
Sep 19 12:27:53.808221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.808199    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:53.927751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927655    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.927751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927702    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.927751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927726    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927758    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927786    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927814    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927841    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927866    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927892    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927920    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927955    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.927982    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928012    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928050    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928107    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928132    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cflbc\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928161    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928187    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928227    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928254    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file\") pod \"66f21207-8b46-4d1c-8fdb-64752d2779eb\" (UID: \"66f21207-8b46-4d1c-8fdb-64752d2779eb\") "
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928265    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.928531 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928305    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.928979 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928536    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.928979 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928901    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-metrics-client-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:53.928979 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928922    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:53.928979 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.928938    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:53.930219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.929278    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:27:53.930219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.929671    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.930397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.930278    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.930397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.930311    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.930848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.930786    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:53.930948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.930881    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.932464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932425    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:27:53.932582 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932458    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out" (OuterVolumeSpecName: "config-out") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:27:53.932778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932746    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.932778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932751    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.932933 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932833    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.932991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.932921    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.933123 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.933099    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.933442 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.933413    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc" (OuterVolumeSpecName: "kube-api-access-cflbc") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "kube-api-access-cflbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:27:53.933484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.933451    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.933909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.933891    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config" (OuterVolumeSpecName: "config") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:53.942777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:53.942751    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config" (OuterVolumeSpecName: "web-config") pod "66f21207-8b46-4d1c-8fdb-64752d2779eb" (UID: "66f21207-8b46-4d1c-8fdb-64752d2779eb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:54.029365 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029335    2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-metrics-client-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029365 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029359    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-db\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029365 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029370    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f21207-8b46-4d1c-8fdb-64752d2779eb-config-out\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029378    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029388    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029396    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029404    2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-config\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029414    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029422    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-tls-assets\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029429    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f21207-8b46-4d1c-8fdb-64752d2779eb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029437    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cflbc\" (UniqueName: \"kubernetes.io/projected/66f21207-8b46-4d1c-8fdb-64752d2779eb-kube-api-access-cflbc\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029445    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029453    2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-grpc-tls\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029462    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-web-config\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029470    2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029477    2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-kube-rbac-proxy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.029601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.029487    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f21207-8b46-4d1c-8fdb-64752d2779eb-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:27:54.409207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.409172    2568 generic.go:356] "Generic (PLEG): container finished" podID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerID="fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225" exitCode=0
Sep 19 12:27:54.409402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.409227    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"}
Sep 19 12:27:54.409402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.409254    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.409402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.409292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f21207-8b46-4d1c-8fdb-64752d2779eb","Type":"ContainerDied","Data":"bf4e51763acd5921ef33954413a183459d8b8a765b03aedc7ba7b3f595805d9c"}
Sep 19 12:27:54.409402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.409326    2568 scope.go:117] "RemoveContainer" containerID="ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"
Sep 19 12:27:54.417668 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.417631    2568 scope.go:117] "RemoveContainer" containerID="3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"
Sep 19 12:27:54.424356 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.424339    2568 scope.go:117] "RemoveContainer" containerID="fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"
Sep 19 12:27:54.430856 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.430842    2568 scope.go:117] "RemoveContainer" containerID="dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"
Sep 19 12:27:54.434768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.434748    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:54.437796 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.437777    2568 scope.go:117] "RemoveContainer" containerID="f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"
Sep 19 12:27:54.441609 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.441588    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:54.444297 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.444265    2568 scope.go:117] "RemoveContainer" containerID="d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"
Sep 19 12:27:54.451048 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.451034    2568 scope.go:117] "RemoveContainer" containerID="e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3"
Sep 19 12:27:54.457207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457194    2568 scope.go:117] "RemoveContainer" containerID="ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"
Sep 19 12:27:54.457449 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.457431    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85\": container with ID starting with ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85 not found: ID does not exist" containerID="ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"
Sep 19 12:27:54.457505 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457457    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85"} err="failed to get container status \"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85\": rpc error: code = NotFound desc = could not find container \"ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85\": container with ID starting with ac48b157aafc352523bea89f4701e38e10427a5dadb737eaa40763715076ba85 not found: ID does not exist"
Sep 19 12:27:54.457505 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457488    2568 scope.go:117] "RemoveContainer" containerID="3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"
Sep 19 12:27:54.457734 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.457717    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692\": container with ID starting with 3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692 not found: ID does not exist" containerID="3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"
Sep 19 12:27:54.457771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457741    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692"} err="failed to get container status \"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692\": rpc error: code = NotFound desc = could not find container \"3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692\": container with ID starting with 3004b84266a36034ffaf9f1fbc3166d54c27d7623510528fd1186269afa72692 not found: ID does not exist"
Sep 19 12:27:54.457771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457759    2568 scope.go:117] "RemoveContainer" containerID="fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"
Sep 19 12:27:54.457993 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.457976    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225\": container with ID starting with fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225 not found: ID does not exist" containerID="fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"
Sep 19 12:27:54.458034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.457998    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225"} err="failed to get container status \"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225\": rpc error: code = NotFound desc = could not find container \"fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225\": container with ID starting with fa6a456cfe7c8943a5f1e01d708c54d6665680748a43ff26ff738fc5990d6225 not found: ID does not exist"
Sep 19 12:27:54.458034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458017    2568 scope.go:117] "RemoveContainer" containerID="dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"
Sep 19 12:27:54.458215 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.458199    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f\": container with ID starting with dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f not found: ID does not exist" containerID="dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"
Sep 19 12:27:54.458254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458219    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f"} err="failed to get container status \"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f\": rpc error: code = NotFound desc = could not find container \"dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f\": container with ID starting with dd80eacf7a959020c7020e768db3921d8d987784af75a53fab5f4c09a0e4097f not found: ID does not exist"
Sep 19 12:27:54.458254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458232    2568 scope.go:117] "RemoveContainer" containerID="f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"
Sep 19 12:27:54.458426 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.458412    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db\": container with ID starting with f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db not found: ID does not exist" containerID="f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"
Sep 19 12:27:54.458464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458429    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db"} err="failed to get container status \"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db\": rpc error: code = NotFound desc = could not find container \"f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db\": container with ID starting with f4491dc6b7f52a46feab5bde49776f1c32adca40bc31bab9e07ebe5b9aa4c5db not found: ID does not exist"
Sep 19 12:27:54.458464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458440    2568 scope.go:117] "RemoveContainer" containerID="d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"
Sep 19 12:27:54.458702 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.458686    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef\": container with ID starting with d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef not found: ID does not exist" containerID="d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"
Sep 19 12:27:54.458753 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458704    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef"} err="failed to get container status \"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef\": rpc error: code = NotFound desc = could not find container \"d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef\": container with ID starting with d45392be0be44f084373490bc7e1f1e0dd72ae8c59517c56ba62a5d4fc1d93ef not found: ID does not exist"
Sep 19 12:27:54.458753 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458718    2568 scope.go:117] "RemoveContainer" containerID="e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3"
Sep 19 12:27:54.458924 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:27:54.458910    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3\": container with ID starting with e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3 not found: ID does not exist" containerID="e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3"
Sep 19 12:27:54.458958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.458926    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3"} err="failed to get container status \"e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3\": rpc error: code = NotFound desc = could not find container \"e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3\": container with ID starting with e8946b575bea930eb27944fd5fe4a5ac52171f5a62a31393a5d326a3fb6b7ed3 not found: ID does not exist"
Sep 19 12:27:54.471447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.471415    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:54.471965 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.471946    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.471970    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.471991    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="prometheus"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472000    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="prometheus"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472027    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="init-config-reloader"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472036    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="init-config-reloader"
Sep 19 12:27:54.472056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472047    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="thanos-sidecar"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472061    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="thanos-sidecar"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472075    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="config-reloader"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472084    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="config-reloader"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472094    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-web"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472103    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-web"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472113    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-thanos"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472127    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-thanos"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472137    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985487cc-b211-4aa0-98b7-1022f4298175" containerName="registry"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472146    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="985487cc-b211-4aa0-98b7-1022f4298175" containerName="registry"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472248    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="config-reloader"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472258    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-web"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472273    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="prometheus"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472281    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472289    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="kube-rbac-proxy-thanos"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472298    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" containerName="thanos-sidecar"
Sep 19 12:27:54.472313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.472313    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="985487cc-b211-4aa0-98b7-1022f4298175" containerName="registry"
Sep 19 12:27:54.476574 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.476557    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.479177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479154    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 19 12:27:54.479177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479169    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 19 12:27:54.479332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479181    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 19 12:27:54.479332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479166    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-j3ju59n0r68f\""
Sep 19 12:27:54.479332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479155    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 19 12:27:54.479332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479155    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 19 12:27:54.479545 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479532    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 19 12:27:54.479598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479584    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 19 12:27:54.479654 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479638    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 19 12:27:54.479743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479726    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 19 12:27:54.479939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479926    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 19 12:27:54.479988 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479945    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 12:27:54.479988 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479931    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 19 12:27:54.480096 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.479979    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wfvb9\""
Sep 19 12:27:54.480147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.480091    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 19 12:27:54.482615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.482564    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 19 12:27:54.485444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.485427    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 19 12:27:54.490135 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.490118    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:54.531860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.531825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.531860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.531864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.531890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.531936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.531991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532063    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44sfn\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-kube-api-access-44sfn\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532152    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532217    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532246    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-web-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532299    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532327    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.532723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.532398    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-config-out\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633078    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44sfn\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-kube-api-access-44sfn\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633194    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633314    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633358    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-web-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633486    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-config-out\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633877    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633923    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.633960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633943    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.634056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.633965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.634056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.634002    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.634598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.634567    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.634759 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.634675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635550 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635533    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635786 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635892 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635876    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.635998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.635982    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636208 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636183    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636228    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636339 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636324    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636566 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636542    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636706 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636678    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.636891 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.636870    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-web-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.637203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.637179    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-config\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.637565 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.637541    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.638560 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.638536    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7527832b-2593-448c-a6eb-01cb401d7105-config-out\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.638659 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.638639    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.638750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.638735    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.638787 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.638776    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7527832b-2593-448c-a6eb-01cb401d7105-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.639366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.639344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7527832b-2593-448c-a6eb-01cb401d7105-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.640979 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.640960    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44sfn\" (UniqueName: \"kubernetes.io/projected/7527832b-2593-448c-a6eb-01cb401d7105-kube-api-access-44sfn\") pod \"prometheus-k8s-0\" (UID: \"7527832b-2593-448c-a6eb-01cb401d7105\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:54.786203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:54.786167    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:27:55.069042 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:55.069016    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 12:27:55.070230 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:27:55.070207    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7527832b_2593_448c_a6eb_01cb401d7105.slice/crio-a4b2220026a18bdb75b00366bcd807598e6638837607401f5e3a01e7a628b901 WatchSource:0}: Error finding container a4b2220026a18bdb75b00366bcd807598e6638837607401f5e3a01e7a628b901: Status 404 returned error can't find the container with id a4b2220026a18bdb75b00366bcd807598e6638837607401f5e3a01e7a628b901
Sep 19 12:27:55.414729 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:55.414691    2568 generic.go:356] "Generic (PLEG): container finished" podID="7527832b-2593-448c-a6eb-01cb401d7105" containerID="155a22a674811a3a270e6664f678684f6280c815d6cbb2b60b59085bfb1bb33d" exitCode=0
Sep 19 12:27:55.414918 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:55.414732    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerDied","Data":"155a22a674811a3a270e6664f678684f6280c815d6cbb2b60b59085bfb1bb33d"}
Sep 19 12:27:55.414918 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:55.414811    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"a4b2220026a18bdb75b00366bcd807598e6638837607401f5e3a01e7a628b901"}
Sep 19 12:27:55.721463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:55.721435    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f21207-8b46-4d1c-8fdb-64752d2779eb" path="/var/lib/kubelet/pods/66f21207-8b46-4d1c-8fdb-64752d2779eb/volumes"
Sep 19 12:27:56.425065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425031    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"c6aef70c76eea6e410c4f5e2edd7273066f99126dc525ce8f65325678e1f649d"}
Sep 19 12:27:56.425463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425121    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"dc6b96a24f8946b1589eae92c036ccd6db62b48e076b3d95920f58427d3087e7"}
Sep 19 12:27:56.425463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425150    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"e4c1e6f1fb23f4f932fb2696617f925063b7739b3e7f514f8efd3e691a896bc7"}
Sep 19 12:27:56.425463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425183    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"aac3aa0ded4a745a8068cce1cc36d1aae5dd7af7020ec3b3a391dc264e200f1d"}
Sep 19 12:27:56.425463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425208    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"a5076722ea47601316c4669892bcd2b3c8f27abe0ef09a28e748f450eb8cd092"}
Sep 19 12:27:56.425463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.425234    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7527832b-2593-448c-a6eb-01cb401d7105","Type":"ContainerStarted","Data":"c6ba6bbf2cf52dc95dab4df5cd9bb393b932e845f24d1816435302267be4a061"}
Sep 19 12:27:56.454809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:56.454757    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.431841658 podStartE2EDuration="2.454741731s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.900073ms" isStatefulPod=false podCreationTimestamp="2026-09-19 12:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:27:56.453456989 +0000 UTC m=+160.743012757" watchObservedRunningTime="2026-09-19 12:27:56.454741731 +0000 UTC m=+160.744297500"
Sep 19 12:27:59.786867 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:27:59.786824    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:28:54.786639 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:28:54.786597    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:28:54.801724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:28:54.801699    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:28:55.659450 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:28:55.659425    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 12:30:17.605739 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:17.605714    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:30:17.612673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:17.612652    2568 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 12:30:17.613372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:17.613353    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:30:32.472755 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.472711    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kngtp"]
Sep 19 12:30:32.475289 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.475274    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.478372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.478350    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 12:30:32.478662 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.478646    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 12:30:32.478758 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.478663    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h4xf2\""
Sep 19 12:30:32.490099 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.490075    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kngtp"]
Sep 19 12:30:32.530246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.530223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-bound-sa-token\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.530366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.530255    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x26c\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-kube-api-access-7x26c\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.630809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.630779    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-bound-sa-token\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.631010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.630821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x26c\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-kube-api-access-7x26c\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.638548 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.638524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-bound-sa-token\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.638704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.638683    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x26c\" (UniqueName: \"kubernetes.io/projected/9486d54c-f11a-451f-accf-c6b9e918ed8a-kube-api-access-7x26c\") pod \"cert-manager-55b4d954d5-kngtp\" (UID: \"9486d54c-f11a-451f-accf-c6b9e918ed8a\") " pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:32.784239 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:32.784151    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kngtp"
Sep 19 12:30:33.073325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:33.073238    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kngtp"]
Sep 19 12:30:33.076197 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:30:33.076166    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9486d54c_f11a_451f_accf_c6b9e918ed8a.slice/crio-59f39eb865cc2cf68cbb310140bac9dd1b5452238b3b920b72b06d35f7b7c110 WatchSource:0}: Error finding container 59f39eb865cc2cf68cbb310140bac9dd1b5452238b3b920b72b06d35f7b7c110: Status 404 returned error can't find the container with id 59f39eb865cc2cf68cbb310140bac9dd1b5452238b3b920b72b06d35f7b7c110
Sep 19 12:30:33.078205 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:33.078191    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:30:33.992490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:33.992453    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kngtp" event={"ID":"9486d54c-f11a-451f-accf-c6b9e918ed8a","Type":"ContainerStarted","Data":"59f39eb865cc2cf68cbb310140bac9dd1b5452238b3b920b72b06d35f7b7c110"}
Sep 19 12:30:37.005853 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:37.005819    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kngtp" event={"ID":"9486d54c-f11a-451f-accf-c6b9e918ed8a","Type":"ContainerStarted","Data":"c7e961c9f292be466b0684b895da6b72ff3e24ac558dc7a543a6fc4fe97aafc9"}
Sep 19 12:30:37.020642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:37.020597    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kngtp" podStartSLOduration=2.037097831 podStartE2EDuration="5.02058357s" totalImagesPullingTime="2.983485739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:30:37.019975398 +0000 UTC m=+321.309531167" watchObservedRunningTime="2026-09-19 12:30:37.02058357 +0000 UTC m=+321.310139329"
Sep 19 12:30:55.309931 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.309889    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"]
Sep 19 12:30:55.312120 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.312098    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.314705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.314684    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 12:30:55.315712 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.315683    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6ms5g\""
Sep 19 12:30:55.315712 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.315702    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 12:30:55.315872 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.315708    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:30:55.315872 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.315750    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 12:30:55.315872 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.315687    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 12:30:55.322044 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.322026    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"]
Sep 19 12:30:55.417189 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.417152    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8b39b0c-e918-4b29-805c-12bdbb78dd51-manager-config\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.417370 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.417232    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlpgr\" (UniqueName: \"kubernetes.io/projected/e8b39b0c-e918-4b29-805c-12bdbb78dd51-kube-api-access-dlpgr\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.417370 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.417266    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.417370 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.417286    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.518223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.518138    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlpgr\" (UniqueName: \"kubernetes.io/projected/e8b39b0c-e918-4b29-805c-12bdbb78dd51-kube-api-access-dlpgr\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.518223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.518175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.518223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.518192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.518519 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.518230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8b39b0c-e918-4b29-805c-12bdbb78dd51-manager-config\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.518934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.518909    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8b39b0c-e918-4b29-805c-12bdbb78dd51-manager-config\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.520638 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.520620    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.520714 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.520644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8b39b0c-e918-4b29-805c-12bdbb78dd51-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.526727 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.526706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlpgr\" (UniqueName: \"kubernetes.io/projected/e8b39b0c-e918-4b29-805c-12bdbb78dd51-kube-api-access-dlpgr\") pod \"lws-controller-manager-6bdf74d45f-4x9p7\" (UID: \"e8b39b0c-e918-4b29-805c-12bdbb78dd51\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.621388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.621354    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:55.894457 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:55.894427    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"]
Sep 19 12:30:55.897249 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:30:55.897220    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b39b0c_e918_4b29_805c_12bdbb78dd51.slice/crio-ce50c02443e8d43098daed86e1201179ec00702b293db7e59ae6406a047dbe4b WatchSource:0}: Error finding container ce50c02443e8d43098daed86e1201179ec00702b293db7e59ae6406a047dbe4b: Status 404 returned error can't find the container with id ce50c02443e8d43098daed86e1201179ec00702b293db7e59ae6406a047dbe4b
Sep 19 12:30:56.078280 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:56.078246    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7" event={"ID":"e8b39b0c-e918-4b29-805c-12bdbb78dd51","Type":"ContainerStarted","Data":"ce50c02443e8d43098daed86e1201179ec00702b293db7e59ae6406a047dbe4b"}
Sep 19 12:30:59.092790 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:59.092752    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7" event={"ID":"e8b39b0c-e918-4b29-805c-12bdbb78dd51","Type":"ContainerStarted","Data":"dfe930c5d9dc5878a420f562c8504cb99131de979ba115915de2dc22a94a41e3"}
Sep 19 12:30:59.093167 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:59.092856    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:30:59.108437 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:30:59.108389    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7" podStartSLOduration=1.68339203 podStartE2EDuration="4.108376285s" totalImagesPullingTime="2.424984255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:30:59.106984281 +0000 UTC m=+343.396540049" watchObservedRunningTime="2026-09-19 12:30:59.108376285 +0000 UTC m=+343.397932053"
Sep 19 12:31:06.859948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.859865    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"]
Sep 19 12:31:06.863085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.863069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:06.865718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.865697    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 12:31:06.865857 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.865725    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 12:31:06.865857 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.865799    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 12:31:06.866374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.866351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bj9t6\""
Sep 19 12:31:06.866473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.866376    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 12:31:06.878014 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.877993    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"]
Sep 19 12:31:06.909015 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.908981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:06.909150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.909022    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc9mq\" (UniqueName: \"kubernetes.io/projected/6aee7517-57b6-474e-af96-b99b73000121-kube-api-access-dc9mq\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:06.909150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.909051    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:06.909150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.909075    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:06.909150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:06.909100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009532 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009480    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009643    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009791 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc9mq\" (UniqueName: \"kubernetes.io/projected/6aee7517-57b6-474e-af96-b99b73000121-kube-api-access-dc9mq\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.009960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.009942    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.011994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.011970    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6aee7517-57b6-474e-af96-b99b73000121-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.012137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.012127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.012211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.012195    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aee7517-57b6-474e-af96-b99b73000121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.017937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.017911    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc9mq\" (UniqueName: \"kubernetes.io/projected/6aee7517-57b6-474e-af96-b99b73000121-kube-api-access-dc9mq\") pod \"opendatahub-operator-controller-manager-774bc4964b-fsxcs\" (UID: \"6aee7517-57b6-474e-af96-b99b73000121\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.172872 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.172841    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:07.450286 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:07.450257    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"]
Sep 19 12:31:07.453070 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:31:07.453045    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aee7517_57b6_474e_af96_b99b73000121.slice/crio-a218e87838f7a62aaf412b0353d22ad46280be2f71d456c59d357bc0bf1763c6 WatchSource:0}: Error finding container a218e87838f7a62aaf412b0353d22ad46280be2f71d456c59d357bc0bf1763c6: Status 404 returned error can't find the container with id a218e87838f7a62aaf412b0353d22ad46280be2f71d456c59d357bc0bf1763c6
Sep 19 12:31:08.127570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:08.127529    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs" event={"ID":"6aee7517-57b6-474e-af96-b99b73000121","Type":"ContainerStarted","Data":"a218e87838f7a62aaf412b0353d22ad46280be2f71d456c59d357bc0bf1763c6"}
Sep 19 12:31:10.099481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:10.099445    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-4x9p7"
Sep 19 12:31:11.143223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:11.143186    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs" event={"ID":"6aee7517-57b6-474e-af96-b99b73000121","Type":"ContainerStarted","Data":"6d900b9825d0d4e69ceedbfb728c94de0846d974e7a95e91cc2a25ac25ad5be4"}
Sep 19 12:31:12.148853 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:12.148814    2568 generic.go:356] "Generic (PLEG): container finished" podID="6aee7517-57b6-474e-af96-b99b73000121" containerID="6d900b9825d0d4e69ceedbfb728c94de0846d974e7a95e91cc2a25ac25ad5be4" exitCode=0
Sep 19 12:31:12.148853 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:12.148855    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs" event={"ID":"6aee7517-57b6-474e-af96-b99b73000121","Type":"ContainerDied","Data":"6d900b9825d0d4e69ceedbfb728c94de0846d974e7a95e91cc2a25ac25ad5be4"}
Sep 19 12:31:13.154461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:13.154430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs" event={"ID":"6aee7517-57b6-474e-af96-b99b73000121","Type":"ContainerStarted","Data":"f28a3b0e8832c533ee42de1a77b92842f99091a3cbcbd97dda6c120429049a7f"}
Sep 19 12:31:13.154879 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:13.154561    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:13.177559 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:13.177487    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs" podStartSLOduration=2.816410142 podStartE2EDuration="7.177466704s" totalImagesPullingTime="3.342803594s" totalInitContainerRuntime="1.018252968s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:31:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:31:13.17494234 +0000 UTC m=+357.464498109" watchObservedRunningTime="2026-09-19 12:31:13.177466704 +0000 UTC m=+357.467022474"
Sep 19 12:31:24.161434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:24.161401    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-fsxcs"
Sep 19 12:31:33.843518 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.837881    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"]
Sep 19 12:31:33.858592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.853876    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.858592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.857861    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 12:31:33.858592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.858181    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 12:31:33.858592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.858478    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tc5v9\""
Sep 19 12:31:33.858937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.858765    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 12:31:33.884285 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.881261    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"]
Sep 19 12:31:33.926463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926423    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926463 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926463    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926619    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f0e431-f923-40a1-b0d3-6466b2621066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gzcd\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-kube-api-access-5gzcd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926839 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926749    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:33.926839 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:33.926777    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027350    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gzcd\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-kube-api-access-5gzcd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027415    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027432    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027560    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027600    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027622    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f0e431-f923-40a1-b0d3-6466b2621066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027859    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.027998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.027900    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.028099 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.028040    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.028099 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.028084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.028318 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.028296    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6f0e431-f923-40a1-b0d3-6466b2621066-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.029995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.029973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.029995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.029979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.037776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.037747    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.037973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.037954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gzcd\" (UniqueName: \"kubernetes.io/projected/b6f0e431-f923-40a1-b0d3-6466b2621066-kube-api-access-5gzcd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt\" (UID: \"b6f0e431-f923-40a1-b0d3-6466b2621066\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.192300 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.192264    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:34.465820 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:34.465654    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"]
Sep 19 12:31:34.468350 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:31:34.468310    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f0e431_f923_40a1_b0d3_6466b2621066.slice/crio-a74649da3e5741912572932ff85c124b210b3c895f417e777b43548dbebaa4cf WatchSource:0}: Error finding container a74649da3e5741912572932ff85c124b210b3c895f417e777b43548dbebaa4cf: Status 404 returned error can't find the container with id a74649da3e5741912572932ff85c124b210b3c895f417e777b43548dbebaa4cf
Sep 19 12:31:35.235831 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:35.235795    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt" event={"ID":"b6f0e431-f923-40a1-b0d3-6466b2621066","Type":"ContainerStarted","Data":"a74649da3e5741912572932ff85c124b210b3c895f417e777b43548dbebaa4cf"}
Sep 19 12:31:37.358184 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:37.358146    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:31:37.358469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:37.358222    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:31:37.358469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:37.358250    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:31:38.250760 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:38.250727    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt" event={"ID":"b6f0e431-f923-40a1-b0d3-6466b2621066","Type":"ContainerStarted","Data":"16eabb9a03a8bf354c7ca0694a4237ed4c6bf79ecbd227fe96b4ca3faad86025"}
Sep 19 12:31:38.271541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:38.271476    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt" podStartSLOduration=2.383865085 podStartE2EDuration="5.271459594s" totalImagesPullingTime="2.887594509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:31:38.269964222 +0000 UTC m=+382.559519990" watchObservedRunningTime="2026-09-19 12:31:38.271459594 +0000 UTC m=+382.561015363"
Sep 19 12:31:39.193330 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:39.193291    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:39.198086 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:39.198059    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:39.255371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:39.255334    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:39.256381 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:39.256360    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt"
Sep 19 12:31:43.871062 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.871030    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"]
Sep 19 12:31:43.877306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.877287    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:43.880658 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.880639    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 19 12:31:43.880916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.880902    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9mz9k\""
Sep 19 12:31:43.881201 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.881182    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 19 12:31:43.881511 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.881479    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 19 12:31:43.928468 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:43.928435    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"]
Sep 19 12:31:44.011193 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011161    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011193 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011197    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc824c90-83a8-49e1-a2c6-80ae32dae915-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvpzv\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-kube-api-access-nvpzv\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011276    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.011458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.011422    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.111829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111794    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.111829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111829    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111857    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc824c90-83a8-49e1-a2c6-80ae32dae915-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111904    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvpzv\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-kube-api-access-nvpzv\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111922    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.111949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.112697 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.112668    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.114256 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.114230    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc824c90-83a8-49e1-a2c6-80ae32dae915-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.114441 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.114421    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.114719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.114701    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.114995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.114978    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.125440 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.125420    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.126880 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.126858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvpzv\" (UniqueName: \"kubernetes.io/projected/dc824c90-83a8-49e1-a2c6-80ae32dae915-kube-api-access-nvpzv\") pod \"istiod-openshift-gateway-5677ccb49b-fr2wz\" (UID: \"dc824c90-83a8-49e1-a2c6-80ae32dae915\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.186349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.186318    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:31:44.480684 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:44.480605    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"]
Sep 19 12:31:44.492765 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:31:44.492739    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc824c90_83a8_49e1_a2c6_80ae32dae915.slice/crio-d7b1d8a9d6dd83b66cd00777d95c15aa172a2764393a380b7b8246a37ec7d3ac WatchSource:0}: Error finding container d7b1d8a9d6dd83b66cd00777d95c15aa172a2764393a380b7b8246a37ec7d3ac: Status 404 returned error can't find the container with id d7b1d8a9d6dd83b66cd00777d95c15aa172a2764393a380b7b8246a37ec7d3ac
Sep 19 12:31:45.282338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:31:45.282298    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz" event={"ID":"dc824c90-83a8-49e1-a2c6-80ae32dae915","Type":"ContainerStarted","Data":"d7b1d8a9d6dd83b66cd00777d95c15aa172a2764393a380b7b8246a37ec7d3ac"}
Sep 19 12:32:00.292399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:00.292361    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:32:00.292670 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:00.292438    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:32:01.354537 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:01.354503    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz" event={"ID":"dc824c90-83a8-49e1-a2c6-80ae32dae915","Type":"ContainerStarted","Data":"2aeecca3fec1382ce4741f2ad7523062612b2ade051a3bb0cf607d1a6bd044fa"}
Sep 19 12:32:01.354935 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:01.354714    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:32:01.356412 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:01.356392    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz"
Sep 19 12:32:01.374131 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:32:01.374084    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fr2wz" podStartSLOduration=2.576814699 podStartE2EDuration="18.374069153s" totalImagesPullingTime="15.797254454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:32:01.371982362 +0000 UTC m=+405.661538108" watchObservedRunningTime="2026-09-19 12:32:01.374069153 +0000 UTC m=+405.663624923"
Sep 19 12:33:45.540709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.540664    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"]
Sep 19 12:33:45.543478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.543454    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.545674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.545656    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-cwxgm\""
Sep 19 12:33:45.555668 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.555643    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"]
Sep 19 12:33:45.600336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600307    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600546 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600340    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgfnf\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-kube-api-access-bgfnf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600546 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600367    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600546 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600424    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600546 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600586    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.600768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.600697    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701644 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701604    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701845 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701650    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgfnf\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-kube-api-access-bgfnf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701845 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701774    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701845 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701833    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.701998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701942    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.701999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702009    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702036    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702335 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702313    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702377    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702420    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.702513 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.702453    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.704385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.704362    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.704594 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.704576    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.709244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.709217    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.709346 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.709310    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgfnf\" (UniqueName: \"kubernetes.io/projected/fbb6d78f-90df-4ed6-b633-7735ba4e54fb-kube-api-access-bgfnf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-sc2x2\" (UID: \"fbb6d78f-90df-4ed6-b633-7735ba4e54fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:45.855585 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:45.855463    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:46.127727 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.127682    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"]
Sep 19 12:33:46.131353 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:33:46.131326    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb6d78f_90df_4ed6_b633_7735ba4e54fb.slice/crio-74fb427c0350484bb905434105b3675833d8fe5828b6e9ccadcf55638a5ceeca WatchSource:0}: Error finding container 74fb427c0350484bb905434105b3675833d8fe5828b6e9ccadcf55638a5ceeca: Status 404 returned error can't find the container with id 74fb427c0350484bb905434105b3675833d8fe5828b6e9ccadcf55638a5ceeca
Sep 19 12:33:46.133398 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.133369    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:33:46.133456 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.133435    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:33:46.133510 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.133465    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:33:46.778447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.778413    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" event={"ID":"fbb6d78f-90df-4ed6-b633-7735ba4e54fb","Type":"ContainerStarted","Data":"b97d02fb4f536aac70d2a7287cc0e7e1a7c6bb4122944e17a6b03d8ff53221a7"}
Sep 19 12:33:46.778854 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.778461    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" event={"ID":"fbb6d78f-90df-4ed6-b633-7735ba4e54fb","Type":"ContainerStarted","Data":"74fb427c0350484bb905434105b3675833d8fe5828b6e9ccadcf55638a5ceeca"}
Sep 19 12:33:46.797765 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.797716    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podStartSLOduration=1.797700547 podStartE2EDuration="1.797700547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:33:46.795959264 +0000 UTC m=+511.085515046" watchObservedRunningTime="2026-09-19 12:33:46.797700547 +0000 UTC m=+511.087256315"
Sep 19 12:33:46.855942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:46.855906    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:47.861837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:47.861808    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:48.787004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:48.786975    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:33:48.788094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:33:48.788070    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:08.410345 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.410315    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"]
Sep 19 12:35:08.413278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.413259    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.415551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.415530    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vvzvj\""
Sep 19 12:35:08.415664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.415529    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 19 12:35:08.423023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.423001    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"]
Sep 19 12:35:08.510338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.510301    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f4550-872a-4378-bd11-16c871a3b369-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.510517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.510375    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8pg9\" (UniqueName: \"kubernetes.io/projected/980f4550-872a-4378-bd11-16c871a3b369-kube-api-access-r8pg9\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.610788 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.610751    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f4550-872a-4378-bd11-16c871a3b369-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.610976 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.610800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8pg9\" (UniqueName: \"kubernetes.io/projected/980f4550-872a-4378-bd11-16c871a3b369-kube-api-access-r8pg9\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.611530 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.611488    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f4550-872a-4378-bd11-16c871a3b369-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.618405 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.618380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8pg9\" (UniqueName: \"kubernetes.io/projected/980f4550-872a-4378-bd11-16c871a3b369-kube-api-access-r8pg9\") pod \"payload-processing-5b54b7fbb7-9qmjq\" (UID: \"980f4550-872a-4378-bd11-16c871a3b369\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:08.724658 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:08.724569    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:09.030521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:09.030478    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"]
Sep 19 12:35:09.031939 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:35:09.031915    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980f4550_872a_4378_bd11_16c871a3b369.slice/crio-3f914354850e121673eed6580d4285669d81126bb96311d007f0282d26111c98 WatchSource:0}: Error finding container 3f914354850e121673eed6580d4285669d81126bb96311d007f0282d26111c98: Status 404 returned error can't find the container with id 3f914354850e121673eed6580d4285669d81126bb96311d007f0282d26111c98
Sep 19 12:35:09.115606 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:09.115568    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq" event={"ID":"980f4550-872a-4378-bd11-16c871a3b369","Type":"ContainerStarted","Data":"3f914354850e121673eed6580d4285669d81126bb96311d007f0282d26111c98"}
Sep 19 12:35:11.126734 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:11.126683    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq" event={"ID":"980f4550-872a-4378-bd11-16c871a3b369","Type":"ContainerStarted","Data":"2a57f54f880a36f1a361f255e19113d62db4a8618c0b7cad114ea85f6db344a0"}
Sep 19 12:35:11.127188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:11.126779    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:11.143779 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:11.143720    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq" podStartSLOduration=1.826741 podStartE2EDuration="3.143700728s" totalImagesPullingTime="1.316959728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:35:11.141463007 +0000 UTC m=+595.431018776" watchObservedRunningTime="2026-09-19 12:35:11.143700728 +0000 UTC m=+595.433256499"
Sep 19 12:35:17.633693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:17.633664    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:35:17.638632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:17.638606    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:35:18.156528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.156475    2568 generic.go:356] "Generic (PLEG): container finished" podID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerID="b97d02fb4f536aac70d2a7287cc0e7e1a7c6bb4122944e17a6b03d8ff53221a7" exitCode=0
Sep 19 12:35:18.156709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.156548    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" event={"ID":"fbb6d78f-90df-4ed6-b633-7735ba4e54fb","Type":"ContainerDied","Data":"b97d02fb4f536aac70d2a7287cc0e7e1a7c6bb4122944e17a6b03d8ff53221a7"}
Sep 19 12:35:18.156965 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.156947    2568 scope.go:117] "RemoveContainer" containerID="b97d02fb4f536aac70d2a7287cc0e7e1a7c6bb4122944e17a6b03d8ff53221a7"
Sep 19 12:35:18.157908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.157880    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:35:18.157963 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.157950    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:35:18.158038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.157983    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:35:18.787734 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:18.787677    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:19.163832 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:19.163802    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" event={"ID":"fbb6d78f-90df-4ed6-b633-7735ba4e54fb","Type":"ContainerStarted","Data":"4d1998b0ceed0b9ff2acd62ed61a6cdca58712e758c0802f3d37156b9c141939"}
Sep 19 12:35:19.855764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:19.855731    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:19.856832 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:19.856809    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-sc2x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 19 12:35:19.856942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:19.856855    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podUID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 19 12:35:20.856113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:20.856072    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-sc2x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 19 12:35:20.856487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:20.856127    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podUID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 19 12:35:21.855862 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:21.855826    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-sc2x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 19 12:35:21.856038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:21.855879    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podUID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 19 12:35:22.133153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:22.133118    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9qmjq"
Sep 19 12:35:22.855962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:22.855924    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-sc2x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 19 12:35:22.856144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:22.855984    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podUID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 19 12:35:23.855883 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:23.855854    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-sc2x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 19 12:35:23.856244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:23.855930    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2" podUID="fbb6d78f-90df-4ed6-b633-7735ba4e54fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 19 12:35:24.860744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:24.860716    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:24.861187 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:24.861092    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:24.861841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:24.861821    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-sc2x2"
Sep 19 12:35:41.120480 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.120404    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"]
Sep 19 12:35:41.123794 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.123775    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.127415 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.127396    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pr82q\""
Sep 19 12:35:41.127629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.127611    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 12:35:41.127711 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.127631    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 19 12:35:41.127795 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.127778    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 12:35:41.135331 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.135299    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"]
Sep 19 12:35:41.199425 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199381    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.199618 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199444    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.199618 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199512    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.199618 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199583    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11bd19f3-fc86-4eee-a138-72338a4417ca-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.199790 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.199790 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.199680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qvtn\" (UniqueName: \"kubernetes.io/projected/11bd19f3-fc86-4eee-a138-72338a4417ca-kube-api-access-8qvtn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300418 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300613 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11bd19f3-fc86-4eee-a138-72338a4417ca-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300613 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300454    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300613 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qvtn\" (UniqueName: \"kubernetes.io/projected/11bd19f3-fc86-4eee-a138-72338a4417ca-kube-api-access-8qvtn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300613 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300613 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300553    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300935 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300990 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300950    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.300990 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.300983    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.302722 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.302701    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11bd19f3-fc86-4eee-a138-72338a4417ca-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.302846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.302828    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11bd19f3-fc86-4eee-a138-72338a4417ca-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.308522 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.308474    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qvtn\" (UniqueName: \"kubernetes.io/projected/11bd19f3-fc86-4eee-a138-72338a4417ca-kube-api-access-8qvtn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw\" (UID: \"11bd19f3-fc86-4eee-a138-72338a4417ca\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.434955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.434925    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:41.702007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.701936    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"]
Sep 19 12:35:41.705256 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:35:41.705229    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bd19f3_fc86_4eee_a138_72338a4417ca.slice/crio-be79e08caddfba2ccaf22203760eeaddac5eb23cf2684a91180b0760ee4b178d WatchSource:0}: Error finding container be79e08caddfba2ccaf22203760eeaddac5eb23cf2684a91180b0760ee4b178d: Status 404 returned error can't find the container with id be79e08caddfba2ccaf22203760eeaddac5eb23cf2684a91180b0760ee4b178d
Sep 19 12:35:41.706592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:41.706569    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:35:42.282192 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:42.282158    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw" event={"ID":"11bd19f3-fc86-4eee-a138-72338a4417ca","Type":"ContainerStarted","Data":"be79e08caddfba2ccaf22203760eeaddac5eb23cf2684a91180b0760ee4b178d"}
Sep 19 12:35:43.287845 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:43.287808    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw" event={"ID":"11bd19f3-fc86-4eee-a138-72338a4417ca","Type":"ContainerStarted","Data":"f4a519ec5010570e07e46e40a7b9c934b219b911ef156b1438bcfef60ef12e9b"}
Sep 19 12:35:43.305836 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:43.305788    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw" podStartSLOduration=1.351283121 podStartE2EDuration="2.305771208s" totalImagesPullingTime="954.488087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:35:43.303748114 +0000 UTC m=+627.593303862" watchObservedRunningTime="2026-09-19 12:35:43.305771208 +0000 UTC m=+627.595326977"
Sep 19 12:35:51.435467 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:51.435421    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:51.435958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:51.435486    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:51.448008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:51.447981    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:52.340287 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:52.340256    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw"
Sep 19 12:35:58.230245 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.230204    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"]
Sep 19 12:35:58.233560 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.233544    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.235694 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.235667    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 19 12:35:58.236617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.236704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236630    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghlfh\" (UniqueName: \"kubernetes.io/projected/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-kube-api-access-ghlfh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.236704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236677    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.236704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.236818 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.236897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.236817    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.243192 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.243173    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"]
Sep 19 12:35:58.337257 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337227    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337257 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337259    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghlfh\" (UniqueName: \"kubernetes.io/projected/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-kube-api-access-ghlfh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337450    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337478    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337689 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337646    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337712    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.337832 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.337810    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.339635 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.339619    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.339835 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.339818    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.344435 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.344413    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghlfh\" (UniqueName: \"kubernetes.io/projected/05ba6e4b-51ca-4bed-8f1c-956a41e83c18-kube-api-access-ghlfh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l\" (UID: \"05ba6e4b-51ca-4bed-8f1c-956a41e83c18\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.543627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.543533    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:35:58.815829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:58.815749    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"]
Sep 19 12:35:58.818895 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:35:58.818866    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ba6e4b_51ca_4bed_8f1c_956a41e83c18.slice/crio-e5b7fb9ef23ef68b23d661a405d4a8a4c08f933bb9a05c32b513e53fd11b3918 WatchSource:0}: Error finding container e5b7fb9ef23ef68b23d661a405d4a8a4c08f933bb9a05c32b513e53fd11b3918: Status 404 returned error can't find the container with id e5b7fb9ef23ef68b23d661a405d4a8a4c08f933bb9a05c32b513e53fd11b3918
Sep 19 12:35:59.359596 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:59.359562    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l" event={"ID":"05ba6e4b-51ca-4bed-8f1c-956a41e83c18","Type":"ContainerStarted","Data":"409c86e06a85354412c3df1883642e29680d82c73cbaa8f852e9013d5617a836"}
Sep 19 12:35:59.359970 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:59.359618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l" event={"ID":"05ba6e4b-51ca-4bed-8f1c-956a41e83c18","Type":"ContainerStarted","Data":"e5b7fb9ef23ef68b23d661a405d4a8a4c08f933bb9a05c32b513e53fd11b3918"}
Sep 19 12:35:59.376350 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:35:59.376300    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l" podStartSLOduration=1.213207347 podStartE2EDuration="1.376284021s" totalImagesPullingTime="163.076674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:35:59.375323967 +0000 UTC m=+643.664879745" watchObservedRunningTime="2026-09-19 12:35:59.376284021 +0000 UTC m=+643.665839790"
Sep 19 12:36:08.544323 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:36:08.544287    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:36:08.544733 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:36:08.544344    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:36:08.556864 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:36:08.556842    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:36:09.413539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:36:09.413507    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l"
Sep 19 12:38:01.072248 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.072214    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:38:01.075472 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.075447    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.078207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.078183    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-61583b-openshift-default-dockercfg-l59ph\""
Sep 19 12:38:01.088044 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.087994    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:38:01.243347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243363    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwh7t\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243394    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243415    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243447    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243479    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.243702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.243556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.345028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.344928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.345028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.344984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.345028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.345020    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.345336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.345061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.345831 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.345799    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwh7t\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.346517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.346627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346538    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.346627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346602    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.346715 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.346936 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346179    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.347033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346124    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.347033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.346355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.347255 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.347231    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.347504 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.347471    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.348526 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.348464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.349354 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.349246    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.354588 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.354516    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwh7t\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.355284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.355261    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token\") pod \"e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") " pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.386285 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.386255    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:01.663582 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.663426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:38:01.666307 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:01.666280    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e30b02_fbee_40fb_aa51_ba48f1e1acf6.slice/crio-3473b2650069a3b6969c93df7f5764d8b48a97f38e33ac7d08c43638347e5a1f WatchSource:0}: Error finding container 3473b2650069a3b6969c93df7f5764d8b48a97f38e33ac7d08c43638347e5a1f: Status 404 returned error can't find the container with id 3473b2650069a3b6969c93df7f5764d8b48a97f38e33ac7d08c43638347e5a1f
Sep 19 12:38:01.668543 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.668483    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:01.668614 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.668583    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:01.668661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.668618    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:01.903607 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.903570    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerStarted","Data":"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"}
Sep 19 12:38:01.903800 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.903633    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerStarted","Data":"3473b2650069a3b6969c93df7f5764d8b48a97f38e33ac7d08c43638347e5a1f"}
Sep 19 12:38:01.923700 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:01.923598    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podStartSLOduration=0.923582965 podStartE2EDuration="923.582965ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:01.921288463 +0000 UTC m=+766.210844233" watchObservedRunningTime="2026-09-19 12:38:01.923582965 +0000 UTC m=+766.213138733"
Sep 19 12:38:02.387025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:02.386997    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:02.392089 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:02.392064    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:02.908396 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:02.908366    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:02.909364 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:02.909349    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:05.764186 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.764140    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:38:05.767579 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.767554    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.770237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.770213    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-61583b\""
Sep 19 12:38:05.770351 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.770267    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-61583b-dockercfg-8xl5d\""
Sep 19 12:38:05.777027 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.776995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:38:05.796512 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.796468    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:38:05.799994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.799972    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:05.806223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.806189    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:38:05.886365 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.886324    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.886800 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.886772    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtx9k\" (UniqueName: \"kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.987911 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.987872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j885p\" (UniqueName: \"kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:05.988118 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.987949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:05.988118 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.988058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtx9k\" (UniqueName: \"kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.988254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.988174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.989185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.989150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:05.996296 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:05.996267    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtx9k\" (UniqueName: \"kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k\") pod \"payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:06.081807 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.081721    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:06.088922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.088899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.089043 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.088993    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j885p\" (UniqueName: \"kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.089624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.089602    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.098472 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.098443    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j885p\" (UniqueName: \"kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p\") pod \"payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.114478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.114431    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.371569 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.371477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:38:06.379803 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:06.379768    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90e03e1_efe7_4d6a_9436_feb4c4840fd7.slice/crio-da94a017abd28d48de747eb7e7d7bb87e61635c2016c1388ae1c7a00e14593d5 WatchSource:0}: Error finding container da94a017abd28d48de747eb7e7d7bb87e61635c2016c1388ae1c7a00e14593d5: Status 404 returned error can't find the container with id da94a017abd28d48de747eb7e7d7bb87e61635c2016c1388ae1c7a00e14593d5
Sep 19 12:38:06.406375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.406354    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:38:06.413696 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:06.413666    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88da8cce_4e41_4a3f_ab34_c4679e6d2bbd.slice/crio-47af366a6c0e0657dec11cae82038b7d7d9b836c8e7a4e9cab7551e070004113 WatchSource:0}: Error finding container 47af366a6c0e0657dec11cae82038b7d7d9b836c8e7a4e9cab7551e070004113: Status 404 returned error can't find the container with id 47af366a6c0e0657dec11cae82038b7d7d9b836c8e7a4e9cab7551e070004113
Sep 19 12:38:06.929056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.929010    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" event={"ID":"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd","Type":"ContainerStarted","Data":"80daf47148d6f23ab6567befc0dc57ff61138ab2f1bc0ba028d3063de56673d4"}
Sep 19 12:38:06.929842 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.929118    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:06.929842 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.929143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" event={"ID":"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd","Type":"ContainerStarted","Data":"47af366a6c0e0657dec11cae82038b7d7d9b836c8e7a4e9cab7551e070004113"}
Sep 19 12:38:06.930839 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.930812    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" event={"ID":"f90e03e1-efe7-4d6a-9436-feb4c4840fd7","Type":"ContainerStarted","Data":"2cf273d622804e953ebac17fcc7d6c833363ddee6fa45475cc6ac65d790a6370"}
Sep 19 12:38:06.930963 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.930860    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" event={"ID":"f90e03e1-efe7-4d6a-9436-feb4c4840fd7","Type":"ContainerStarted","Data":"da94a017abd28d48de747eb7e7d7bb87e61635c2016c1388ae1c7a00e14593d5"}
Sep 19 12:38:06.930963 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.930906    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:06.945060 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.945005    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" podStartSLOduration=1.944986386 podStartE2EDuration="1.944986386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:06.944395159 +0000 UTC m=+771.233950942" watchObservedRunningTime="2026-09-19 12:38:06.944986386 +0000 UTC m=+771.234542158"
Sep 19 12:38:06.961859 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:06.961811    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" podStartSLOduration=1.9617925920000001 podStartE2EDuration="1.961792592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:06.959694446 +0000 UTC m=+771.249250194" watchObservedRunningTime="2026-09-19 12:38:06.961792592 +0000 UTC m=+771.251348366"
Sep 19 12:38:09.365207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.365167    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:38:09.369890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.369869    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.372267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.372242    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-a2e924\""
Sep 19 12:38:09.372376 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.372317    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-a2e924-dockercfg-vh6m8\""
Sep 19 12:38:09.381579 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.381556    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:38:09.400232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.400204    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:38:09.403602 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.403579    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.411912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.411893    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:38:09.523298 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.523262    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.523480 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.523322    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.523480 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.523356    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jxs7\" (UniqueName: \"kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.523480 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.523424    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfv58\" (UniqueName: \"kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.624597 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.624473    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.624597 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.624559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxs7\" (UniqueName: \"kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.624836 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.624613    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfv58\" (UniqueName: \"kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.624836 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.624676    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.625287 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.625252    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.625379 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.625361    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.632425 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.632398    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jxs7\" (UniqueName: \"kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7\") pod \"payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.632557 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.632398    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfv58\" (UniqueName: \"kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58\") pod \"payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.682152 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.682125    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:09.714428 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.714380    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:09.960517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.960461    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:38:09.963246 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:09.963217    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2047b98b_b833_4c06_9b8d_057de409e66b.slice/crio-f7a71d339dfb53773e0754c79dfa59ccce03e0d48a31509ef097e18f5a1eaddb WatchSource:0}: Error finding container f7a71d339dfb53773e0754c79dfa59ccce03e0d48a31509ef097e18f5a1eaddb: Status 404 returned error can't find the container with id f7a71d339dfb53773e0754c79dfa59ccce03e0d48a31509ef097e18f5a1eaddb
Sep 19 12:38:09.995886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:09.995862    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:38:09.999567 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:09.999538    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0a14ea_668c_4473_9409_efd271900763.slice/crio-b86d3d0f453eb843c02bbccaaad2525ce9496dc23bb2b61df9b2fecd90f36305 WatchSource:0}: Error finding container b86d3d0f453eb843c02bbccaaad2525ce9496dc23bb2b61df9b2fecd90f36305: Status 404 returned error can't find the container with id b86d3d0f453eb843c02bbccaaad2525ce9496dc23bb2b61df9b2fecd90f36305
Sep 19 12:38:10.459783 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.459749    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:10.463672 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.463581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.468128 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.468106    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5f80f73c-openshift-default-dockercfg-rlklr\""
Sep 19 12:38:10.475402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.475373    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:10.534166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534134    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534168    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534400 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534400 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534264    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534400 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534324    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534400 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534376    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534415    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjpb7\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.534621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.534487    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.635545 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.635490    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.635794 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.635776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.635943 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.635918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.635920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636084    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjpb7\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636283 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636283 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636224    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636409    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636653 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636628    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636899 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636870    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.636899 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.636885    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.637535 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.637512    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.638398 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.638373    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.639130 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.639108    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.647057 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.647036    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.647645 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.647625    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjpb7\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7\") pod \"e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") " pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.781751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.781670    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:10.953443 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.953411    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" event={"ID":"2047b98b-b833-4c06-9b8d-057de409e66b","Type":"ContainerStarted","Data":"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"}
Sep 19 12:38:10.953657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.953489    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" event={"ID":"2047b98b-b833-4c06-9b8d-057de409e66b","Type":"ContainerStarted","Data":"f7a71d339dfb53773e0754c79dfa59ccce03e0d48a31509ef097e18f5a1eaddb"}
Sep 19 12:38:10.953657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.953550    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:10.955080 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.955052    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" event={"ID":"0b0a14ea-668c-4473-9409-efd271900763","Type":"ContainerStarted","Data":"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"}
Sep 19 12:38:10.955197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.955104    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" event={"ID":"0b0a14ea-668c-4473-9409-efd271900763","Type":"ContainerStarted","Data":"b86d3d0f453eb843c02bbccaaad2525ce9496dc23bb2b61df9b2fecd90f36305"}
Sep 19 12:38:10.955197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.955132    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:10.971394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.971341    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" podStartSLOduration=1.971319131 podStartE2EDuration="1.971319131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:10.97012751 +0000 UTC m=+775.259683280" watchObservedRunningTime="2026-09-19 12:38:10.971319131 +0000 UTC m=+775.260874903"
Sep 19 12:38:10.987086 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:10.987038    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" podStartSLOduration=1.98702236 podStartE2EDuration="1.98702236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:10.986406567 +0000 UTC m=+775.275962336" watchObservedRunningTime="2026-09-19 12:38:10.98702236 +0000 UTC m=+775.276578148"
Sep 19 12:38:11.059397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.059376    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:11.061208 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:11.061175    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d42994_e89c_4a36_9c56_2e3766e5c1f3.slice/crio-2f5201cdd0db0dd335bce1cf32309a1bb16abfd5538bd7337a54b6365a8ddb48 WatchSource:0}: Error finding container 2f5201cdd0db0dd335bce1cf32309a1bb16abfd5538bd7337a54b6365a8ddb48: Status 404 returned error can't find the container with id 2f5201cdd0db0dd335bce1cf32309a1bb16abfd5538bd7337a54b6365a8ddb48
Sep 19 12:38:11.063562 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.063479    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:11.063687 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.063610    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:11.063747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.063674    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:11.961741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.961702    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerStarted","Data":"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"}
Sep 19 12:38:11.962166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.961756    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerStarted","Data":"2f5201cdd0db0dd335bce1cf32309a1bb16abfd5538bd7337a54b6365a8ddb48"}
Sep 19 12:38:11.983782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:11.983732    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podStartSLOduration=1.9837164330000001 podStartE2EDuration="1.983716433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:11.981982036 +0000 UTC m=+776.271537804" watchObservedRunningTime="2026-09-19 12:38:11.983716433 +0000 UTC m=+776.273272202"
Sep 19 12:38:12.782743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:12.782704    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:12.788109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:12.788085    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:12.966566 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:12.966536    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:12.967605 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:12.967588    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:17.342223 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.342188    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:17.346307 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.346284    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.348750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.348727    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f80f73c-dockercfg-gl94d\""
Sep 19 12:38:17.348859 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.348753    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5f80f73c\""
Sep 19 12:38:17.360018 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.359990    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:17.404188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.404142    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95pwq\" (UniqueName: \"kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.404392 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.404268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.505047 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.504998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95pwq\" (UniqueName: \"kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.505251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.505141    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.506142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.506104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.515539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.512931    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95pwq\" (UniqueName: \"kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq\") pod \"payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") " pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.665025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.664797    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f80f73c-dockercfg-gl94d\""
Sep 19 12:38:17.673524 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.670577    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:17.938212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.938129    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:38:17.938212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.938193    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:38:17.979036 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:17.977917    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:18.009899 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.009861    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" event={"ID":"9328c799-f420-40e8-834d-c0bc8b06391d","Type":"ContainerStarted","Data":"9ccaf02b5fc7381025a08fc68bde7a8cad79c6b2b4bd2c454be04942fe5254d8"}
Sep 19 12:38:18.593377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.593330    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:38:18.597435 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.597406    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.600011 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.599927    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-38aa0a-dockercfg-4tftm\""
Sep 19 12:38:18.600221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.600204    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-38aa0a\""
Sep 19 12:38:18.605508 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.605445    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:38:18.616796 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.616696    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.616939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.616806    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfkz2\" (UniqueName: \"kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.718264 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.718224    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.718461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.718310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfkz2\" (UniqueName: \"kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.718970 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.718951    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.725912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.725881    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfkz2\" (UniqueName: \"kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2\") pod \"payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") " pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:18.911393 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:18.911356    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:19.018305 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:19.018274    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" event={"ID":"9328c799-f420-40e8-834d-c0bc8b06391d","Type":"ContainerStarted","Data":"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"}
Sep 19 12:38:19.018691 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:19.018670    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:19.041127 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:19.041054    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" podStartSLOduration=2.041034641 podStartE2EDuration="2.041034641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:19.039302973 +0000 UTC m=+783.328858744" watchObservedRunningTime="2026-09-19 12:38:19.041034641 +0000 UTC m=+783.330590411"
Sep 19 12:38:19.236041 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:19.236004    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:38:19.238292 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:19.238267    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bfb5ca5_d749_425e_ad05_02877541c17b.slice/crio-38407558052b666784e80326bb08a193f4f3ee85b85a8ce35339084bc2eb5b3d WatchSource:0}: Error finding container 38407558052b666784e80326bb08a193f4f3ee85b85a8ce35339084bc2eb5b3d: Status 404 returned error can't find the container with id 38407558052b666784e80326bb08a193f4f3ee85b85a8ce35339084bc2eb5b3d
Sep 19 12:38:20.024922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:20.024886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" event={"ID":"6bfb5ca5-d749-425e-ad05-02877541c17b","Type":"ContainerStarted","Data":"ad856bdb46ee641feb96f5037f387261b51d5c252299e4ee7d030586325e9e9b"}
Sep 19 12:38:20.025417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:20.024943    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" event={"ID":"6bfb5ca5-d749-425e-ad05-02877541c17b","Type":"ContainerStarted","Data":"38407558052b666784e80326bb08a193f4f3ee85b85a8ce35339084bc2eb5b3d"}
Sep 19 12:38:20.044600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:20.044550    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" podStartSLOduration=2.04453519 podStartE2EDuration="2.04453519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:20.041894108 +0000 UTC m=+784.331449895" watchObservedRunningTime="2026-09-19 12:38:20.04453519 +0000 UTC m=+784.334090958"
Sep 19 12:38:21.030539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:21.030491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:21.964412 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:21.964376    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:38:21.964608 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:21.964482    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:38:24.046996 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.046967    2568 generic.go:356] "Generic (PLEG): container finished" podID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerID="fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654" exitCode=0
Sep 19 12:38:24.047417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.047026    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerDied","Data":"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"}
Sep 19 12:38:24.047461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.047418    2568 scope.go:117] "RemoveContainer" containerID="fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"
Sep 19 12:38:24.048598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.048568    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:24.048681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.048642    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:24.048725 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.048683    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:24.172001 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.171928    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:38:24.174510 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.174482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.177360 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.177340    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5be349\""
Sep 19 12:38:24.185593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.185568    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:38:24.197396 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.197370    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:38:24.199768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.199746    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.202109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.202090    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5be349-dockercfg-q4424\""
Sep 19 12:38:24.210844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.210292    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:38:24.278536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.278485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.278704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.278607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.278704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.278634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvj86\" (UniqueName: \"kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.278704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.278656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpjf2\" (UniqueName: \"kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.380006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.379976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.380171 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.380016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvj86\" (UniqueName: \"kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.380171 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.380042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpjf2\" (UniqueName: \"kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.380171 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.380154    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.380810 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.380786    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.380922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.380905    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.388474 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.388444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpjf2\" (UniqueName: \"kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2\") pod \"payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.388623 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.388590    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvj86\" (UniqueName: \"kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86\") pod \"payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.488185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.488149    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:24.514472 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.514443    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:24.814258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.814233    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:38:24.816707 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:24.816681    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb2647a_5179_4b44_bda7_59962133486e.slice/crio-e502e165cde4bca5ebecb1f31a43bf30cec0e78866ffd5b65bbe51243f74431c WatchSource:0}: Error finding container e502e165cde4bca5ebecb1f31a43bf30cec0e78866ffd5b65bbe51243f74431c: Status 404 returned error can't find the container with id e502e165cde4bca5ebecb1f31a43bf30cec0e78866ffd5b65bbe51243f74431c
Sep 19 12:38:24.836018 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:24.835995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:38:24.839937 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:24.839906    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e3f6ee_b6ec_4899_897a_b81eab56dc3f.slice/crio-b324de44ed60f2b3659ea0f96700f18d4842bcc5955ce317bd656bd1ff39a665 WatchSource:0}: Error finding container b324de44ed60f2b3659ea0f96700f18d4842bcc5955ce317bd656bd1ff39a665: Status 404 returned error can't find the container with id b324de44ed60f2b3659ea0f96700f18d4842bcc5955ce317bd656bd1ff39a665
Sep 19 12:38:25.054592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.054470    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" event={"ID":"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f","Type":"ContainerStarted","Data":"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"}
Sep 19 12:38:25.054592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.054549    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" event={"ID":"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f","Type":"ContainerStarted","Data":"b324de44ed60f2b3659ea0f96700f18d4842bcc5955ce317bd656bd1ff39a665"}
Sep 19 12:38:25.055080 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.054670    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:25.056721 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.056693    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerStarted","Data":"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"}
Sep 19 12:38:25.058319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.058300    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" event={"ID":"9bb2647a-5179-4b44-bda7-59962133486e","Type":"ContainerStarted","Data":"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"}
Sep 19 12:38:25.058394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.058339    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" event={"ID":"9bb2647a-5179-4b44-bda7-59962133486e","Type":"ContainerStarted","Data":"e502e165cde4bca5ebecb1f31a43bf30cec0e78866ffd5b65bbe51243f74431c"}
Sep 19 12:38:25.058441 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.058414    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:25.072113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.072000    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" podStartSLOduration=1.071978843 podStartE2EDuration="1.071978843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:25.070162698 +0000 UTC m=+789.359718465" watchObservedRunningTime="2026-09-19 12:38:25.071978843 +0000 UTC m=+789.361534616"
Sep 19 12:38:25.111059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.111005    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" podStartSLOduration=1.110984803 podStartE2EDuration="1.110984803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:25.110487192 +0000 UTC m=+789.400042963" watchObservedRunningTime="2026-09-19 12:38:25.110984803 +0000 UTC m=+789.400540575"
Sep 19 12:38:25.386578 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.386528    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:25.387053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.387020    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 19 12:38:25.387144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:25.387080    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 19 12:38:26.386964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:26.386928    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 19 12:38:26.387547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:26.387515    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 19 12:38:27.387846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:27.387369    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 19 12:38:27.387846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:27.387433    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 19 12:38:28.388948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:28.388902    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 19 12:38:28.389426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:28.388975    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 19 12:38:29.387258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:29.387219    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Sep 19 12:38:29.387923 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:29.387890    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Sep 19 12:38:30.465191 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:30.465107    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:30.466324 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:30.466302    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:30.473989 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:30.473965    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:38:31.032511 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:31.032467    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:32.037457 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:32.037421    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:38:33.887017 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.886986    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"]
Sep 19 12:38:33.889403 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.889388    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:33.892139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.892116    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:38:33.892371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.892352    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:38:33.903054 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.903027    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"]
Sep 19 12:38:33.994688 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:33.994648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp\" (UID: \"da0dd7d4-1d30-4123-a32f-403851fbfbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:34.095903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:34.095842    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp\" (UID: \"da0dd7d4-1d30-4123-a32f-403851fbfbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:34.096931 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:34.096879    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp\" (UID: \"da0dd7d4-1d30-4123-a32f-403851fbfbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:34.203680 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:34.203595    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:34.491674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:34.491647    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"]
Sep 19 12:38:34.493974 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:34.493945    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0dd7d4_1d30_4123_a32f_403851fbfbff.slice/crio-c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735 WatchSource:0}: Error finding container c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735: Status 404 returned error can't find the container with id c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735
Sep 19 12:38:35.120461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:35.120421    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp" event={"ID":"da0dd7d4-1d30-4123-a32f-403851fbfbff","Type":"ContainerStarted","Data":"c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735"}
Sep 19 12:38:36.067605 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.067573    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:38:36.068243 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.068223    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:38:36.130058 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.130024    2568 generic.go:356] "Generic (PLEG): container finished" podID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerID="6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97" exitCode=0
Sep 19 12:38:36.130576 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.130101    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerDied","Data":"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"}
Sep 19 12:38:36.130703 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.130677    2568 scope.go:117] "RemoveContainer" containerID="6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"
Sep 19 12:38:36.132521 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.132464    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:36.132637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.132611    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:36.132736 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:36.132706    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:37.139438 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:37.139405    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerStarted","Data":"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"}
Sep 19 12:38:37.782466 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:37.782434    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:37.782651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:37.782609    2568 patch_prober.go:28] interesting pod/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 19 12:38:37.782718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:37.782651    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 19 12:38:38.149425 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:38.149389    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp" event={"ID":"da0dd7d4-1d30-4123-a32f-403851fbfbff","Type":"ContainerStarted","Data":"07c312e8e7503e06ad15ded1e51392921d4239eb3d28cb4569d5b2d777067aa6"}
Sep 19 12:38:38.169477 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:38.169412    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp" podStartSLOduration=2.212331701 podStartE2EDuration="5.169391305s" totalImagesPullingTime="2.957059604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:38.166368137 +0000 UTC m=+802.455923907" watchObservedRunningTime="2026-09-19 12:38:38.169391305 +0000 UTC m=+802.458947075"
Sep 19 12:38:38.783182 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:38.783074    2568 patch_prober.go:28] interesting pod/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 19 12:38:38.783182 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:38.783147    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 19 12:38:39.155958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:39.155928    2568 generic.go:356] "Generic (PLEG): container finished" podID="da0dd7d4-1d30-4123-a32f-403851fbfbff" containerID="07c312e8e7503e06ad15ded1e51392921d4239eb3d28cb4569d5b2d777067aa6" exitCode=0
Sep 19 12:38:39.156334 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:39.156008    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp" event={"ID":"da0dd7d4-1d30-4123-a32f-403851fbfbff","Type":"ContainerDied","Data":"07c312e8e7503e06ad15ded1e51392921d4239eb3d28cb4569d5b2d777067aa6"}
Sep 19 12:38:39.782451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:39.782405    2568 patch_prober.go:28] interesting pod/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 19 12:38:39.782689 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:39.782473    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 19 12:38:40.329408 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.329381    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:40.488015 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.487923    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca\") pod \"da0dd7d4-1d30-4123-a32f-403851fbfbff\" (UID: \"da0dd7d4-1d30-4123-a32f-403851fbfbff\") "
Sep 19 12:38:40.488260 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.488235    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da0dd7d4-1d30-4123-a32f-403851fbfbff" (UID: "da0dd7d4-1d30-4123-a32f-403851fbfbff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:38:40.588966 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.588929    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0dd7d4-1d30-4123-a32f-403851fbfbff-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:40.782206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.782106    2568 patch_prober.go:28] interesting pod/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 19 12:38:40.782206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:40.782178    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 19 12:38:41.172147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:41.172115    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"
Sep 19 12:38:41.172714 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:41.172115    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp" event={"ID":"da0dd7d4-1d30-4123-a32f-403851fbfbff","Type":"ContainerDied","Data":"c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735"}
Sep 19 12:38:41.172927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:41.172897    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4524f5a6d7d9f760926fe8635b955d90c45771cdee6563622de5ba250805735"
Sep 19 12:38:41.783153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:41.782713    2568 patch_prober.go:28] interesting pod/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 19 12:38:41.783153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:41.782778    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 19 12:38:42.786581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:42.786552    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:42.786968 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:42.786942    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:42.787620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:42.787601    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:44.652890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.652859    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:44.654143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.653136    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" podUID="9328c799-f420-40e8-834d-c0bc8b06391d" containerName="payload-processing" containerID="cri-o://28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25" gracePeriod=30
Sep 19 12:38:44.858295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.858210    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"]
Sep 19 12:38:44.865004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.864973    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f80f73c-fl6vp"]
Sep 19 12:38:44.914065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.913995    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:44.972222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:44.972191    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:45.035474 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.035443    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95pwq\" (UniqueName: \"kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq\") pod \"9328c799-f420-40e8-834d-c0bc8b06391d\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") "
Sep 19 12:38:45.035697 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.035562    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume\") pod \"9328c799-f420-40e8-834d-c0bc8b06391d\" (UID: \"9328c799-f420-40e8-834d-c0bc8b06391d\") "
Sep 19 12:38:45.035945 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.035919    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9328c799-f420-40e8-834d-c0bc8b06391d" (UID: "9328c799-f420-40e8-834d-c0bc8b06391d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:38:45.037783 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.037751    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq" (OuterVolumeSpecName: "kube-api-access-95pwq") pod "9328c799-f420-40e8-834d-c0bc8b06391d" (UID: "9328c799-f420-40e8-834d-c0bc8b06391d"). InnerVolumeSpecName "kube-api-access-95pwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:38:45.136679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.136641    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95pwq\" (UniqueName: \"kubernetes.io/projected/9328c799-f420-40e8-834d-c0bc8b06391d-kube-api-access-95pwq\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.136679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.136670    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9328c799-f420-40e8-834d-c0bc8b06391d-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.195666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195579    2568 generic.go:356] "Generic (PLEG): container finished" podID="9328c799-f420-40e8-834d-c0bc8b06391d" containerID="28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25" exitCode=0
Sep 19 12:38:45.195666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195628    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"
Sep 19 12:38:45.195893 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195661    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" event={"ID":"9328c799-f420-40e8-834d-c0bc8b06391d","Type":"ContainerDied","Data":"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"}
Sep 19 12:38:45.195893 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195716    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm" event={"ID":"9328c799-f420-40e8-834d-c0bc8b06391d","Type":"ContainerDied","Data":"9ccaf02b5fc7381025a08fc68bde7a8cad79c6b2b4bd2c454be04942fe5254d8"}
Sep 19 12:38:45.195893 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195750    2568 scope.go:117] "RemoveContainer" containerID="28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"
Sep 19 12:38:45.196039 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.195982    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy" containerID="cri-o://a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e" gracePeriod=30
Sep 19 12:38:45.205107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.205083    2568 scope.go:117] "RemoveContainer" containerID="28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"
Sep 19 12:38:45.205412 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:38:45.205394    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25\": container with ID starting with 28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25 not found: ID does not exist" containerID="28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"
Sep 19 12:38:45.205489 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.205420    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25"} err="failed to get container status \"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25\": rpc error: code = NotFound desc = could not find container \"28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25\": container with ID starting with 28f658abee4f7ab7444732c565da3d18fe9e5894aec11a5c1d1044150414be25 not found: ID does not exist"
Sep 19 12:38:45.231743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.231717    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:45.236218 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.236194    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f80f73c-78584b9b59-mvrmm"]
Sep 19 12:38:45.726741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.724652    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9328c799-f420-40e8-834d-c0bc8b06391d" path="/var/lib/kubelet/pods/9328c799-f420-40e8-834d-c0bc8b06391d/volumes"
Sep 19 12:38:45.726741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:45.725187    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da0dd7d4-1d30-4123-a32f-403851fbfbff" path="/var/lib/kubelet/pods/da0dd7d4-1d30-4123-a32f-403851fbfbff/volumes"
Sep 19 12:38:50.447536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.447512    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:50.587987 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.587908    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.587987 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.587948    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.587992    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588029    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588053    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjpb7\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588131    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588256    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588302    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588304    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588315    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588330    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket\") pod \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\" (UID: \"f2d42994-e89c-4a36-9c56-2e3766e5c1f3\") "
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588332    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:38:50.588404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588352    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:38:50.588895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588664    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:38:50.588895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588723    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.588895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588745    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.588895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588758    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.588895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.588770    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.590355 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.590330    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:38:50.590355 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.590353    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:38:50.590481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.590337    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:38:50.590481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.590338    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7" (OuterVolumeSpecName: "kube-api-access-cjpb7") pod "f2d42994-e89c-4a36-9c56-2e3766e5c1f3" (UID: "f2d42994-e89c-4a36-9c56-2e3766e5c1f3"). InnerVolumeSpecName "kube-api-access-cjpb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:38:50.689903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.689875    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.689903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.689901    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.689903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.689911    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.690112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.689922    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:50.690112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:50.689934    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjpb7\" (UniqueName: \"kubernetes.io/projected/f2d42994-e89c-4a36-9c56-2e3766e5c1f3-kube-api-access-cjpb7\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:38:51.227384 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.227352    2568 generic.go:356] "Generic (PLEG): container finished" podID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerID="a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e" exitCode=0
Sep 19 12:38:51.227570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.227403    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerDied","Data":"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"}
Sep 19 12:38:51.227570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.227416    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"
Sep 19 12:38:51.227570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.227446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv" event={"ID":"f2d42994-e89c-4a36-9c56-2e3766e5c1f3","Type":"ContainerDied","Data":"2f5201cdd0db0dd335bce1cf32309a1bb16abfd5538bd7337a54b6365a8ddb48"}
Sep 19 12:38:51.227570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.227472    2568 scope.go:117] "RemoveContainer" containerID="a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"
Sep 19 12:38:51.237309 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.237289    2568 scope.go:117] "RemoveContainer" containerID="6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"
Sep 19 12:38:51.245176 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.245148    2568 scope.go:117] "RemoveContainer" containerID="a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"
Sep 19 12:38:51.245448 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:38:51.245426    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e\": container with ID starting with a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e not found: ID does not exist" containerID="a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"
Sep 19 12:38:51.245548 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.245454    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e"} err="failed to get container status \"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e\": rpc error: code = NotFound desc = could not find container \"a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e\": container with ID starting with a61dfb1dd7a68898dd1ba33d7c9dde4c2dd9ec278fa7ea75e09db80c01a34a4e not found: ID does not exist"
Sep 19 12:38:51.245548 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.245470    2568 scope.go:117] "RemoveContainer" containerID="6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"
Sep 19 12:38:51.245705 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:38:51.245689    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97\": container with ID starting with 6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97 not found: ID does not exist" containerID="6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"
Sep 19 12:38:51.245745 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.245709    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97"} err="failed to get container status \"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97\": rpc error: code = NotFound desc = could not find container \"6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97\": container with ID starting with 6dfda1fd08388ed7d00255a270975528d9cee4539ae583a52f6b18974dbf4c97 not found: ID does not exist"
Sep 19 12:38:51.253508 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.253464    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:51.258222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.258197    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5f80f73c-openshift-default-6dbfc99d7b-n5rtv"]
Sep 19 12:38:51.722530 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:51.722475    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" path="/var/lib/kubelet/pods/f2d42994-e89c-4a36-9c56-2e3766e5c1f3/volumes"
Sep 19 12:38:56.002243 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002214    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:38:56.002661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002640    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:38:56.002661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002652    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:38:56.002661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002663    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0dd7d4-1d30-4123-a32f-403851fbfbff" containerName="revoke-keys"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002668    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0dd7d4-1d30-4123-a32f-403851fbfbff" containerName="revoke-keys"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002681    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9328c799-f420-40e8-834d-c0bc8b06391d" containerName="payload-processing"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002687    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9328c799-f420-40e8-834d-c0bc8b06391d" containerName="payload-processing"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002741    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="da0dd7d4-1d30-4123-a32f-403851fbfbff" containerName="revoke-keys"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002748    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002753    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9328c799-f420-40e8-834d-c0bc8b06391d" containerName="payload-processing"
Sep 19 12:38:56.002773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.002762    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:38:56.005807 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.005787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.008686 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.008669    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9b376ee6-dockercfg-hlf7s\""
Sep 19 12:38:56.008962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.008947    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9b376ee6\""
Sep 19 12:38:56.017145 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.017124    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:38:56.040153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.040129    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk5nh\" (UniqueName: \"kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.040289 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.040164    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.141056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.141021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk5nh\" (UniqueName: \"kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.141211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.141063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.141727 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.141709    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.149401 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.149374    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk5nh\" (UniqueName: \"kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh\") pod \"payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.315949 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.315873    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:56.609509 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:56.609460    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:38:56.611697 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:38:56.611657    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3e72ba_5f30_41f0_8a6c_960a1ec64cee.slice/crio-bbc35086dffa091d719b2f0cee11cc54118cdc07727754a0c3484966e895a2c9 WatchSource:0}: Error finding container bbc35086dffa091d719b2f0cee11cc54118cdc07727754a0c3484966e895a2c9: Status 404 returned error can't find the container with id bbc35086dffa091d719b2f0cee11cc54118cdc07727754a0c3484966e895a2c9
Sep 19 12:38:57.259038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:57.259005    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" event={"ID":"de3e72ba-5f30-41f0-8a6c-960a1ec64cee","Type":"ContainerStarted","Data":"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"}
Sep 19 12:38:57.259038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:57.259053    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" event={"ID":"de3e72ba-5f30-41f0-8a6c-960a1ec64cee","Type":"ContainerStarted","Data":"bbc35086dffa091d719b2f0cee11cc54118cdc07727754a0c3484966e895a2c9"}
Sep 19 12:38:57.259751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:57.259091    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:38:57.278391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:38:57.278336    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" podStartSLOduration=2.278314912 podStartE2EDuration="2.278314912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:57.274631802 +0000 UTC m=+821.564187567" watchObservedRunningTime="2026-09-19 12:38:57.278314912 +0000 UTC m=+821.567870681"
Sep 19 12:39:08.267307 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:08.267277    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:39:20.628083 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.628048    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"]
Sep 19 12:39:20.628470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.628442    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:39:20.628470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.628454    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d42994-e89c-4a36-9c56-2e3766e5c1f3" containerName="istio-proxy"
Sep 19 12:39:20.633935 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.633916    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:20.636443 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.636413    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:39:20.636707 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.636688    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:39:20.637712 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.637689    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"]
Sep 19 12:39:20.766837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.766805    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9\" (UID: \"18fcca30-9a3c-4d13-ba7e-0615cb05ac74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:20.868145 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.868108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9\" (UID: \"18fcca30-9a3c-4d13-ba7e-0615cb05ac74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:20.868761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.868739    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9\" (UID: \"18fcca30-9a3c-4d13-ba7e-0615cb05ac74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:20.946063 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:20.945989    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:21.213121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:21.213089    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"]
Sep 19 12:39:21.215396 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:39:21.215373    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18fcca30_9a3c_4d13_ba7e_0615cb05ac74.slice/crio-477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c WatchSource:0}: Error finding container 477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c: Status 404 returned error can't find the container with id 477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c
Sep 19 12:39:21.381406 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:21.381376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9" event={"ID":"18fcca30-9a3c-4d13-ba7e-0615cb05ac74","Type":"ContainerStarted","Data":"477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c"}
Sep 19 12:39:22.387506 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:22.387459    2568 generic.go:356] "Generic (PLEG): container finished" podID="18fcca30-9a3c-4d13-ba7e-0615cb05ac74" containerID="ab67aa1a31fae827c2f34d60efecae85d4953d7fc898f6f1d6ec48ea3ad68806" exitCode=0
Sep 19 12:39:22.387898 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:22.387531    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9" event={"ID":"18fcca30-9a3c-4d13-ba7e-0615cb05ac74","Type":"ContainerDied","Data":"ab67aa1a31fae827c2f34d60efecae85d4953d7fc898f6f1d6ec48ea3ad68806"}
Sep 19 12:39:23.585807 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:23.585785    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:23.693213 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:23.693130    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca\") pod \"18fcca30-9a3c-4d13-ba7e-0615cb05ac74\" (UID: \"18fcca30-9a3c-4d13-ba7e-0615cb05ac74\") "
Sep 19 12:39:23.693478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:23.693455    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18fcca30-9a3c-4d13-ba7e-0615cb05ac74" (UID: "18fcca30-9a3c-4d13-ba7e-0615cb05ac74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:39:23.794548 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:23.794484    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18fcca30-9a3c-4d13-ba7e-0615cb05ac74-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:39:24.398249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:24.398213    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9" event={"ID":"18fcca30-9a3c-4d13-ba7e-0615cb05ac74","Type":"ContainerDied","Data":"477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c"}
Sep 19 12:39:24.398249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:24.398230    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"
Sep 19 12:39:24.398489 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:24.398261    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="477fc8b0dc7315c9251b7258ebf091e570218007a4ba3f42943afaa636e5979c"
Sep 19 12:39:32.477722 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.477688    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:39:32.478506 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.478451    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" podUID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" containerName="payload-processing" containerID="cri-o://d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670" gracePeriod=30
Sep 19 12:39:32.643978 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.643913    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"]
Sep 19 12:39:32.646938 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.646578    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9b376ee6-757d9"]
Sep 19 12:39:32.737969 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.737893    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:39:32.780321 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.780266    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk5nh\" (UniqueName: \"kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh\") pod \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") "
Sep 19 12:39:32.783397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.783365    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh" (OuterVolumeSpecName: "kube-api-access-fk5nh") pod "de3e72ba-5f30-41f0-8a6c-960a1ec64cee" (UID: "de3e72ba-5f30-41f0-8a6c-960a1ec64cee"). InnerVolumeSpecName "kube-api-access-fk5nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:39:32.881369 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.881333    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume\") pod \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\" (UID: \"de3e72ba-5f30-41f0-8a6c-960a1ec64cee\") "
Sep 19 12:39:32.881643 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.881625    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk5nh\" (UniqueName: \"kubernetes.io/projected/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-kube-api-access-fk5nh\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.881730 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.881662    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de3e72ba-5f30-41f0-8a6c-960a1ec64cee" (UID: "de3e72ba-5f30-41f0-8a6c-960a1ec64cee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:39:32.982775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:32.982746    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de3e72ba-5f30-41f0-8a6c-960a1ec64cee-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:39:33.441863 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.441833    2568 generic.go:356] "Generic (PLEG): container finished" podID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" containerID="d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670" exitCode=0
Sep 19 12:39:33.442051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.441874    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"
Sep 19 12:39:33.442051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.441901    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" event={"ID":"de3e72ba-5f30-41f0-8a6c-960a1ec64cee","Type":"ContainerDied","Data":"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"}
Sep 19 12:39:33.442051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.441953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5" event={"ID":"de3e72ba-5f30-41f0-8a6c-960a1ec64cee","Type":"ContainerDied","Data":"bbc35086dffa091d719b2f0cee11cc54118cdc07727754a0c3484966e895a2c9"}
Sep 19 12:39:33.442051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.441979    2568 scope.go:117] "RemoveContainer" containerID="d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"
Sep 19 12:39:33.450736 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.450716    2568 scope.go:117] "RemoveContainer" containerID="d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"
Sep 19 12:39:33.451017 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:39:33.451001    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670\": container with ID starting with d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670 not found: ID does not exist" containerID="d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"
Sep 19 12:39:33.451073 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.451023    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670"} err="failed to get container status \"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670\": rpc error: code = NotFound desc = could not find container \"d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670\": container with ID starting with d31c3e0cf8bb1a562e2a689fd37347ff7b20c22639cef48c299fe8362c9c4670 not found: ID does not exist"
Sep 19 12:39:33.466912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.466889    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:39:33.475363 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.475339    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b376ee6-7ccc596594-cdqj5"]
Sep 19 12:39:33.490522 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.490477    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:39:33.491045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491024    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" containerName="payload-processing"
Sep 19 12:39:33.491121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491048    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" containerName="payload-processing"
Sep 19 12:39:33.491121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491060    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fcca30-9a3c-4d13-ba7e-0615cb05ac74" containerName="revoke-keys"
Sep 19 12:39:33.491121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491068    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fcca30-9a3c-4d13-ba7e-0615cb05ac74" containerName="revoke-keys"
Sep 19 12:39:33.491270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491156    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" containerName="payload-processing"
Sep 19 12:39:33.491270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.491169    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fcca30-9a3c-4d13-ba7e-0615cb05ac74" containerName="revoke-keys"
Sep 19 12:39:33.495565 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.495547    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.498048 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.498029    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-450ffc30-openshift-default-dockercfg-kb7fc\""
Sep 19 12:39:33.510261 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.510240    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587059    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587156    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587281    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.587528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587529    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt5ht\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.588137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.588137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587623    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.588137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.587652    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688698 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688652    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688698 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688736    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688792    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt5ht\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.688927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.688945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.689301    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.689324    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.689385    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.689551    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.689583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.689577    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.691019 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.690992    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.691335 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.691317    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.700253 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.700195    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt5ht\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.700426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.700406    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token\") pod \"e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") " pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:33.722159 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.722133    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18fcca30-9a3c-4d13-ba7e-0615cb05ac74" path="/var/lib/kubelet/pods/18fcca30-9a3c-4d13-ba7e-0615cb05ac74/volumes"
Sep 19 12:39:33.722487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.722475    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3e72ba-5f30-41f0-8a6c-960a1ec64cee" path="/var/lib/kubelet/pods/de3e72ba-5f30-41f0-8a6c-960a1ec64cee/volumes"
Sep 19 12:39:33.805813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:33.805786    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:34.086634 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.086486    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:39:34.089465 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:39:34.089436    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae63c797_4f4e_4fce_ab67_d6ad9c1740e2.slice/crio-f4af89c0d3108932f8f1eac3b0a0ed6d3aaec5d7f7c0760f25fb9d3b797b16a4 WatchSource:0}: Error finding container f4af89c0d3108932f8f1eac3b0a0ed6d3aaec5d7f7c0760f25fb9d3b797b16a4: Status 404 returned error can't find the container with id f4af89c0d3108932f8f1eac3b0a0ed6d3aaec5d7f7c0760f25fb9d3b797b16a4
Sep 19 12:39:34.092149 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.092116    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:34.092244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.092197    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:34.092305 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.092241    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:34.449590 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.449559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerStarted","Data":"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"}
Sep 19 12:39:34.449751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.449609    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerStarted","Data":"f4af89c0d3108932f8f1eac3b0a0ed6d3aaec5d7f7c0760f25fb9d3b797b16a4"}
Sep 19 12:39:34.469312 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.469263    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podStartSLOduration=1.469246207 podStartE2EDuration="1.469246207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:34.467292948 +0000 UTC m=+858.756848717" watchObservedRunningTime="2026-09-19 12:39:34.469246207 +0000 UTC m=+858.758802038"
Sep 19 12:39:34.806560 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:34.806475    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:35.811096 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:35.811064    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:36.463022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:36.462994    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:36.463872 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:36.463852    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:39:38.698230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.698196    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:39:38.701816 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.701792    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.704285 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.704268    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-450ffc30-dockercfg-cm8fl\""
Sep 19 12:39:38.704599 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.704586    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-450ffc30\""
Sep 19 12:39:38.714966 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.714943    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:39:38.733117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.731953    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74ngr\" (UniqueName: \"kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.733117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.732037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.832665 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.832625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74ngr\" (UniqueName: \"kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.832836 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.832728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.833432 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.833409    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:38.842012 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:38.841985    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74ngr\" (UniqueName: \"kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr\") pod \"payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:39.012552 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.012458    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:39.289397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.289349    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:39:39.292427 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:39:39.292395    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8c0801_804a_4a5c_a003_62fe5587ba7f.slice/crio-eb383e4aa83bad1064ee6df111a94ca5803e83f7d6c3a015ef93b4810fb848fa WatchSource:0}: Error finding container eb383e4aa83bad1064ee6df111a94ca5803e83f7d6c3a015ef93b4810fb848fa: Status 404 returned error can't find the container with id eb383e4aa83bad1064ee6df111a94ca5803e83f7d6c3a015ef93b4810fb848fa
Sep 19 12:39:39.481763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.481726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" event={"ID":"2c8c0801-804a-4a5c-a003-62fe5587ba7f","Type":"ContainerStarted","Data":"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"}
Sep 19 12:39:39.481932 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.481798    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" event={"ID":"2c8c0801-804a-4a5c-a003-62fe5587ba7f","Type":"ContainerStarted","Data":"eb383e4aa83bad1064ee6df111a94ca5803e83f7d6c3a015ef93b4810fb848fa"}
Sep 19 12:39:39.481932 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.481887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:39:39.500704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:39.500657    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" podStartSLOduration=1.5006410209999999 podStartE2EDuration="1.500641021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:39.49893297 +0000 UTC m=+863.788488739" watchObservedRunningTime="2026-09-19 12:39:39.500641021 +0000 UTC m=+863.790196815"
Sep 19 12:39:50.489682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:39:50.489651    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:40:00.591272 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.591190    2568 generic.go:356] "Generic (PLEG): container finished" podID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerID="c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3" exitCode=0
Sep 19 12:40:00.591794 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.591268    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerDied","Data":"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"}
Sep 19 12:40:00.591861 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.591817    2568 scope.go:117] "RemoveContainer" containerID="c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"
Sep 19 12:40:00.593065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.593038    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:00.593130 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.593103    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:00.593165 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:00.593132    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:01.598155 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:01.598123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerStarted","Data":"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"}
Sep 19 12:40:01.806913 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:01.806883    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:01.807959 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:01.807936    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:01.808071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:01.807978    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:02.806323 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:02.806287    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:02.806791 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:02.806374    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:03.806610 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:03.806570    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:03.806994 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:03.806630    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:04.807148 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:04.807121    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:04.807514 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:04.807169    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:05.806546 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:05.806488    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:05.806721 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:05.806564    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:06.463222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.463183    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:06.619098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.619064    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"]
Sep 19 12:40:06.621515 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.621474    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:06.623937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.623918    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:40:06.624034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.624007    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:40:06.630389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.630368    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"]
Sep 19 12:40:06.710071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.710027    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-450ffc30-qvb85\" (UID: \"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:06.806275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.806190    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 19 12:40:06.806275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.806243    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 19 12:40:06.810743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.810714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-450ffc30-qvb85\" (UID: \"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:06.811541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.811518    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-450ffc30-qvb85\" (UID: \"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:06.932321 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:06.932286    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:07.235470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.235444    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"]
Sep 19 12:40:07.236711 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:40:07.236670    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84adb1a_7e12_40f0_b19f_3d93a9ff1f4e.slice/crio-04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b WatchSource:0}: Error finding container 04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b: Status 404 returned error can't find the container with id 04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b
Sep 19 12:40:07.638558 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.638521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85" event={"ID":"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e","Type":"ContainerStarted","Data":"e87ec8c98a606838296dec8e67b9b7c1bc83a0577a1f61e72e9d537d613b6eaa"}
Sep 19 12:40:07.638933 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.638603    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85" event={"ID":"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e","Type":"ContainerStarted","Data":"04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b"}
Sep 19 12:40:07.654973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.654913    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85" podStartSLOduration=1.654897301 podStartE2EDuration="1.654897301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:07.652786779 +0000 UTC m=+891.942342548" watchObservedRunningTime="2026-09-19 12:40:07.654897301 +0000 UTC m=+891.944453068"
Sep 19 12:40:07.810417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.810389    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:07.811622 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:07.811607    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:08.644799 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:08.644769    2568 generic.go:356] "Generic (PLEG): container finished" podID="a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" containerID="e87ec8c98a606838296dec8e67b9b7c1bc83a0577a1f61e72e9d537d613b6eaa" exitCode=0
Sep 19 12:40:08.645193 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:08.644859    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85" event={"ID":"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e","Type":"ContainerDied","Data":"e87ec8c98a606838296dec8e67b9b7c1bc83a0577a1f61e72e9d537d613b6eaa"}
Sep 19 12:40:09.690841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.690814    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"]
Sep 19 12:40:09.693188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.693171    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:09.701960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.701439    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"]
Sep 19 12:40:09.797627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.797603    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:09.842620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.842593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-38aa0a-lxb89\" (UID: \"2483091c-17d4-4f94-ba1a-caba903198c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:09.943455 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.943373    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca\") pod \"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e\" (UID: \"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e\") "
Sep 19 12:40:09.943643 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.943626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-38aa0a-lxb89\" (UID: \"2483091c-17d4-4f94-ba1a-caba903198c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:09.943796 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.943774    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" (UID: "a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:09.944197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:09.944180    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-38aa0a-lxb89\" (UID: \"2483091c-17d4-4f94-ba1a-caba903198c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:10.006517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.006468    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:10.045037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.045002    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:10.275163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.275139    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"]
Sep 19 12:40:10.277049 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:40:10.277020    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2483091c_17d4_4f94_ba1a_caba903198c9.slice/crio-6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e WatchSource:0}: Error finding container 6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e: Status 404 returned error can't find the container with id 6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e
Sep 19 12:40:10.656109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.656073    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"
Sep 19 12:40:10.656276 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.656069    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85" event={"ID":"a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e","Type":"ContainerDied","Data":"04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b"}
Sep 19 12:40:10.656276 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.656207    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04cd0aad7fd5e944a409efc71c38d3eaaa317aee51bd5cccfca9faa35a84ac5b"
Sep 19 12:40:10.657656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.657635    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89" event={"ID":"2483091c-17d4-4f94-ba1a-caba903198c9","Type":"ContainerStarted","Data":"7b6a5efecd07e8da5b4eae7c8aa1c2465281a07ce02d8b64b93b0b9f4f3c6943"}
Sep 19 12:40:10.657789 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.657676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89" event={"ID":"2483091c-17d4-4f94-ba1a-caba903198c9","Type":"ContainerStarted","Data":"6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e"}
Sep 19 12:40:10.674456 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:10.674415    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89" podStartSLOduration=1.67440092 podStartE2EDuration="1.67440092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:10.672274861 +0000 UTC m=+894.961830630" watchObservedRunningTime="2026-09-19 12:40:10.67440092 +0000 UTC m=+894.963956696"
Sep 19 12:40:11.663779 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:11.663745    2568 generic.go:356] "Generic (PLEG): container finished" podID="2483091c-17d4-4f94-ba1a-caba903198c9" containerID="7b6a5efecd07e8da5b4eae7c8aa1c2465281a07ce02d8b64b93b0b9f4f3c6943" exitCode=0
Sep 19 12:40:11.664140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:11.663824    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89" event={"ID":"2483091c-17d4-4f94-ba1a-caba903198c9","Type":"ContainerDied","Data":"7b6a5efecd07e8da5b4eae7c8aa1c2465281a07ce02d8b64b93b0b9f4f3c6943"}
Sep 19 12:40:12.800361 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:12.800339    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:12.973741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:12.973653    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca\") pod \"2483091c-17d4-4f94-ba1a-caba903198c9\" (UID: \"2483091c-17d4-4f94-ba1a-caba903198c9\") "
Sep 19 12:40:12.973972 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:12.973953    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2483091c-17d4-4f94-ba1a-caba903198c9" (UID: "2483091c-17d4-4f94-ba1a-caba903198c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:13.074846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:13.074811    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2483091c-17d4-4f94-ba1a-caba903198c9-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:13.674787 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:13.674760    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"
Sep 19 12:40:13.674955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:13.674760    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89" event={"ID":"2483091c-17d4-4f94-ba1a-caba903198c9","Type":"ContainerDied","Data":"6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e"}
Sep 19 12:40:13.674955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:13.674880    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d7febaef3b42105e0457718b106fa7eadc8c73735b9bc4a4e1ee82de8ae985e"
Sep 19 12:40:17.667986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:17.667951    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:40:17.670212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:17.670190    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:40:18.146651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.146605    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:40:18.146930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.146874    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" podUID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" containerName="payload-pre-processing" containerID="cri-o://f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3" gracePeriod=30
Sep 19 12:40:18.303209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.303144    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"]
Sep 19 12:40:18.305055 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.305033    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-450ffc30-qvb85"]
Sep 19 12:40:18.430209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.430180    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:40:18.524178 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.524135    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74ngr\" (UniqueName: \"kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr\") pod \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") "
Sep 19 12:40:18.524178 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.524185    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume\") pod \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\" (UID: \"2c8c0801-804a-4a5c-a003-62fe5587ba7f\") "
Sep 19 12:40:18.524662 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.524633    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c8c0801-804a-4a5c-a003-62fe5587ba7f" (UID: "2c8c0801-804a-4a5c-a003-62fe5587ba7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:18.526243 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.526225    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr" (OuterVolumeSpecName: "kube-api-access-74ngr") pod "2c8c0801-804a-4a5c-a003-62fe5587ba7f" (UID: "2c8c0801-804a-4a5c-a003-62fe5587ba7f"). InnerVolumeSpecName "kube-api-access-74ngr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:18.624991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.624961    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74ngr\" (UniqueName: \"kubernetes.io/projected/2c8c0801-804a-4a5c-a003-62fe5587ba7f-kube-api-access-74ngr\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.624991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.624987    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c8c0801-804a-4a5c-a003-62fe5587ba7f-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.700279 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.700195    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" containerID="f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3" exitCode=0
Sep 19 12:40:18.700279 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.700239    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"
Sep 19 12:40:18.700678 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.700275    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" event={"ID":"2c8c0801-804a-4a5c-a003-62fe5587ba7f","Type":"ContainerDied","Data":"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"}
Sep 19 12:40:18.700678 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.700335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr" event={"ID":"2c8c0801-804a-4a5c-a003-62fe5587ba7f","Type":"ContainerDied","Data":"eb383e4aa83bad1064ee6df111a94ca5803e83f7d6c3a015ef93b4810fb848fa"}
Sep 19 12:40:18.700678 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.700362    2568 scope.go:117] "RemoveContainer" containerID="f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"
Sep 19 12:40:18.710010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.709994    2568 scope.go:117] "RemoveContainer" containerID="f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"
Sep 19 12:40:18.710301 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:40:18.710280    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3\": container with ID starting with f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3 not found: ID does not exist" containerID="f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"
Sep 19 12:40:18.710384 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.710307    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3"} err="failed to get container status \"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3\": rpc error: code = NotFound desc = could not find container \"f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3\": container with ID starting with f2cc695a7a3bdf53faa0acdc8571cf7338cfbc2f83133b6b034ab193b45fc7b3 not found: ID does not exist"
Sep 19 12:40:18.721181 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.721148    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:40:18.724793 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:18.724770    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-450ffc30-66bd84b8b6-vlrbr"]
Sep 19 12:40:19.381133 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:19.381101    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:40:19.381349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:19.381324    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" containerID="cri-o://e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04" gracePeriod=30
Sep 19 12:40:19.721228 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:19.721149    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" path="/var/lib/kubelet/pods/2c8c0801-804a-4a5c-a003-62fe5587ba7f/volumes"
Sep 19 12:40:19.721593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:19.721469    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" path="/var/lib/kubelet/pods/a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e/volumes"
Sep 19 12:40:22.517357 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.517328    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:40:22.517886 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.517546    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" podUID="6bfb5ca5-d749-425e-ad05-02877541c17b" containerName="payload-processing" containerID="cri-o://ad856bdb46ee641feb96f5037f387261b51d5c252299e4ee7d030586325e9e9b" gracePeriod=30
Sep 19 12:40:22.685692 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.685623    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"]
Sep 19 12:40:22.688473 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.688442    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-38aa0a-lxb89"]
Sep 19 12:40:22.730508 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.730458    2568 generic.go:356] "Generic (PLEG): container finished" podID="6bfb5ca5-d749-425e-ad05-02877541c17b" containerID="ad856bdb46ee641feb96f5037f387261b51d5c252299e4ee7d030586325e9e9b" exitCode=0
Sep 19 12:40:22.730672 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.730533    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" event={"ID":"6bfb5ca5-d749-425e-ad05-02877541c17b","Type":"ContainerDied","Data":"ad856bdb46ee641feb96f5037f387261b51d5c252299e4ee7d030586325e9e9b"}
Sep 19 12:40:22.789865 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.789832    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:40:22.813229 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.813196    2568 patch_prober.go:28] interesting pod/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 12:40:22.813391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.813258    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:40:22.865926 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.865896    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume\") pod \"6bfb5ca5-d749-425e-ad05-02877541c17b\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") "
Sep 19 12:40:22.866079 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.866009    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfkz2\" (UniqueName: \"kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2\") pod \"6bfb5ca5-d749-425e-ad05-02877541c17b\" (UID: \"6bfb5ca5-d749-425e-ad05-02877541c17b\") "
Sep 19 12:40:22.866328 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.866302    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bfb5ca5-d749-425e-ad05-02877541c17b" (UID: "6bfb5ca5-d749-425e-ad05-02877541c17b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:22.867981 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.867964    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2" (OuterVolumeSpecName: "kube-api-access-gfkz2") pod "6bfb5ca5-d749-425e-ad05-02877541c17b" (UID: "6bfb5ca5-d749-425e-ad05-02877541c17b"). InnerVolumeSpecName "kube-api-access-gfkz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:22.966823 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.966791    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfb5ca5-d749-425e-ad05-02877541c17b-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:22.966823 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:22.966815    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfkz2\" (UniqueName: \"kubernetes.io/projected/6bfb5ca5-d749-425e-ad05-02877541c17b-kube-api-access-gfkz2\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:23.722251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.722215    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2483091c-17d4-4f94-ba1a-caba903198c9" path="/var/lib/kubelet/pods/2483091c-17d4-4f94-ba1a-caba903198c9/volumes"
Sep 19 12:40:23.736481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.736461    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"
Sep 19 12:40:23.736481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.736472    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb" event={"ID":"6bfb5ca5-d749-425e-ad05-02877541c17b","Type":"ContainerDied","Data":"38407558052b666784e80326bb08a193f4f3ee85b85a8ce35339084bc2eb5b3d"}
Sep 19 12:40:23.736679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.736538    2568 scope.go:117] "RemoveContainer" containerID="ad856bdb46ee641feb96f5037f387261b51d5c252299e4ee7d030586325e9e9b"
Sep 19 12:40:23.756680 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.756652    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:40:23.761476 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:23.761450    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-38aa0a-758b6f444f-7b2qb"]
Sep 19 12:40:24.652452 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.652393    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:24.689593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689565    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.689769 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689675    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.689769 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689705    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.689769 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689723    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.689769 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689758    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689788    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689806    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt5ht\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689825    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689856    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy\") pod \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\" (UID: \"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2\") "
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.689979    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:24.690107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690075    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690120    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690195    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690213    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690246    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690254    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:24.690402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.690292    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:24.692252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.692206    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:40:24.692528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.692291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht" (OuterVolumeSpecName: "kube-api-access-mt5ht") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "kube-api-access-mt5ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:24.692528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.692453    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:24.692528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.692516    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" (UID: "ae63c797-4f4e-4fce-ab67-d6ad9c1740e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:24.743368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.743339    2568 generic.go:356] "Generic (PLEG): container finished" podID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerID="e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04" exitCode=0
Sep 19 12:40:24.743803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.743385    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"
Sep 19 12:40:24.743803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.743387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerDied","Data":"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"}
Sep 19 12:40:24.743803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.743445    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9" event={"ID":"ae63c797-4f4e-4fce-ab67-d6ad9c1740e2","Type":"ContainerDied","Data":"f4af89c0d3108932f8f1eac3b0a0ed6d3aaec5d7f7c0760f25fb9d3b797b16a4"}
Sep 19 12:40:24.743803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.743479    2568 scope.go:117] "RemoveContainer" containerID="e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"
Sep 19 12:40:24.752806 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.752790    2568 scope.go:117] "RemoveContainer" containerID="c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"
Sep 19 12:40:24.760841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.760828    2568 scope.go:117] "RemoveContainer" containerID="e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"
Sep 19 12:40:24.761109 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:40:24.761093    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04\": container with ID starting with e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04 not found: ID does not exist" containerID="e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"
Sep 19 12:40:24.761170 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.761115    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04"} err="failed to get container status \"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04\": rpc error: code = NotFound desc = could not find container \"e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04\": container with ID starting with e2d1660d1fab7388c7be6631467305bc7dd5ddaa2b6449a2cf284c923cae1a04 not found: ID does not exist"
Sep 19 12:40:24.761170 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.761140    2568 scope.go:117] "RemoveContainer" containerID="c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"
Sep 19 12:40:24.761365 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:40:24.761349    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3\": container with ID starting with c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3 not found: ID does not exist" containerID="c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"
Sep 19 12:40:24.761405 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.761369    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3"} err="failed to get container status \"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3\": rpc error: code = NotFound desc = could not find container \"c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3\": container with ID starting with c9f4ce8a4fc11ccf2f017ea92a3d8356d2abde50e66b840ad6e478c87aeb37a3 not found: ID does not exist"
Sep 19 12:40:24.766020 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.765998    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:40:24.769455 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.769438    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-450ffc30-openshift-default-7bfcfcfbf8-kvkk9"]
Sep 19 12:40:24.792044 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792024    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.792163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792049    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt5ht\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-kube-api-access-mt5ht\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.792163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792059    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.792163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792069    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.792163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792077    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:24.792163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:24.792084    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:25.722651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:25.722619    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfb5ca5-d749-425e-ad05-02877541c17b" path="/var/lib/kubelet/pods/6bfb5ca5-d749-425e-ad05-02877541c17b/volumes"
Sep 19 12:40:25.723035 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:25.723023    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" path="/var/lib/kubelet/pods/ae63c797-4f4e-4fce-ab67-d6ad9c1740e2/volumes"
Sep 19 12:40:27.336518 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336464    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:40:27.336884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336865    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2483091c-17d4-4f94-ba1a-caba903198c9" containerName="revoke-keys"
Sep 19 12:40:27.336884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336878    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2483091c-17d4-4f94-ba1a-caba903198c9" containerName="revoke-keys"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336890    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" containerName="payload-pre-processing"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336896    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" containerName="payload-pre-processing"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336913    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336923    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336935    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336941    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336948    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfb5ca5-d749-425e-ad05-02877541c17b" containerName="payload-processing"
Sep 19 12:40:27.336958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336953    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfb5ca5-d749-425e-ad05-02877541c17b" containerName="payload-processing"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336974    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" containerName="revoke-keys"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.336980    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" containerName="revoke-keys"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.337039    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8c0801-804a-4a5c-a003-62fe5587ba7f" containerName="payload-pre-processing"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.337047    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.337054    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfb5ca5-d749-425e-ad05-02877541c17b" containerName="payload-processing"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.337060    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2483091c-17d4-4f94-ba1a-caba903198c9" containerName="revoke-keys"
Sep 19 12:40:27.337203 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.337066    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a84adb1a-7e12-40f0-b19f-3d93a9ff1f4e" containerName="revoke-keys"
Sep 19 12:40:27.340031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.340014    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.342470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.342450    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-21b2975e-dockercfg-wgrff\""
Sep 19 12:40:27.342470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.342463    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-21b2975e\""
Sep 19 12:40:27.350339 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.350319    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:40:27.373157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.373133    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:40:27.373544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.373531    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae63c797-4f4e-4fce-ab67-d6ad9c1740e2" containerName="istio-proxy"
Sep 19 12:40:27.375321 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.375308    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.386942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.386919    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:40:27.414579 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.414551    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs2k2\" (UniqueName: \"kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.414714 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.414695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.414777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.414729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.414777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.414753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l62ld\" (UniqueName: \"kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.515929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.515893    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.515929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.515932    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.516202 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.515955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l62ld\" (UniqueName: \"kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.516202 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.516061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs2k2\" (UniqueName: \"kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.516639 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.516616    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.516752 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.516699    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.523656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.523630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l62ld\" (UniqueName: \"kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld\") pod \"payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.523656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.523641    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs2k2\" (UniqueName: \"kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2\") pod \"payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") " pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.650645 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.650613    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:27.684671 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.684638    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:27.930020 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.929999    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:40:27.931909 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:40:27.931884    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d5f16a_899f_4fcb_b949_993c0cc48e73.slice/crio-71be4571e8a4c6634df120448a5be5e71fe1d71e490f25572acb250d16e70de0 WatchSource:0}: Error finding container 71be4571e8a4c6634df120448a5be5e71fe1d71e490f25572acb250d16e70de0: Status 404 returned error can't find the container with id 71be4571e8a4c6634df120448a5be5e71fe1d71e490f25572acb250d16e70de0
Sep 19 12:40:27.985541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:27.985233    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:40:28.769949 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.769909    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" event={"ID":"93d5f16a-899f-4fcb-b949-993c0cc48e73","Type":"ContainerStarted","Data":"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"}
Sep 19 12:40:28.770414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.769964    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" event={"ID":"93d5f16a-899f-4fcb-b949-993c0cc48e73","Type":"ContainerStarted","Data":"71be4571e8a4c6634df120448a5be5e71fe1d71e490f25572acb250d16e70de0"}
Sep 19 12:40:28.770414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.770022    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:28.771670 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.771646    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" event={"ID":"feabcbeb-282b-4c43-83b7-5f4368508edb","Type":"ContainerStarted","Data":"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"}
Sep 19 12:40:28.771827 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.771691    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" event={"ID":"feabcbeb-282b-4c43-83b7-5f4368508edb","Type":"ContainerStarted","Data":"7f4fb104ce44102dfd825a7bee4d66535b50335abe2e768a1e4a02993ab3d51e"}
Sep 19 12:40:28.771827 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.771749    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:28.787250 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.787206    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" podStartSLOduration=1.787194551 podStartE2EDuration="1.787194551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:28.784841369 +0000 UTC m=+913.074397151" watchObservedRunningTime="2026-09-19 12:40:28.787194551 +0000 UTC m=+913.076750389"
Sep 19 12:40:28.801006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:28.800965    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" podStartSLOduration=1.800913395 podStartE2EDuration="1.800913395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:28.800116552 +0000 UTC m=+913.089672332" watchObservedRunningTime="2026-09-19 12:40:28.800913395 +0000 UTC m=+913.090469166"
Sep 19 12:40:39.778973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:39.778939    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:40:39.779447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:39.779002    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:40:53.128359 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.128328    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"]
Sep 19 12:40:53.131121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.131101    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:53.133475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.133452    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:40:53.133585 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.133456    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:40:53.139175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.139157    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"]
Sep 19 12:40:53.160618 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.160591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-21b2975e-mkggk\" (UID: \"f496dda2-72a5-4017-8d58-ac771958f301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:53.261660 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.261630    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-21b2975e-mkggk\" (UID: \"f496dda2-72a5-4017-8d58-ac771958f301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:53.262262 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.262238    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-21b2975e-mkggk\" (UID: \"f496dda2-72a5-4017-8d58-ac771958f301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:53.443153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.443068    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:53.713048 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.712907    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"]
Sep 19 12:40:53.715853 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:40:53.715819    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf496dda2_72a5_4017_8d58_ac771958f301.slice/crio-66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74 WatchSource:0}: Error finding container 66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74: Status 404 returned error can't find the container with id 66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74
Sep 19 12:40:53.898307 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:53.898273    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk" event={"ID":"f496dda2-72a5-4017-8d58-ac771958f301","Type":"ContainerStarted","Data":"66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74"}
Sep 19 12:40:54.904554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:54.904520    2568 generic.go:356] "Generic (PLEG): container finished" podID="f496dda2-72a5-4017-8d58-ac771958f301" containerID="dbb8322b709dad5fd22fd7a1e35cd88d3d2b43fc6c08f0620b45d56b02e22348" exitCode=0
Sep 19 12:40:54.904933 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:54.904572    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk" event={"ID":"f496dda2-72a5-4017-8d58-ac771958f301","Type":"ContainerDied","Data":"dbb8322b709dad5fd22fd7a1e35cd88d3d2b43fc6c08f0620b45d56b02e22348"}
Sep 19 12:40:56.035871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.035844    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:56.083184 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.083153    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca\") pod \"f496dda2-72a5-4017-8d58-ac771958f301\" (UID: \"f496dda2-72a5-4017-8d58-ac771958f301\") "
Sep 19 12:40:56.083562 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.083541    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f496dda2-72a5-4017-8d58-ac771958f301" (UID: "f496dda2-72a5-4017-8d58-ac771958f301"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:56.184229 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.184147    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f496dda2-72a5-4017-8d58-ac771958f301-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:40:56.915879 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.915847    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"
Sep 19 12:40:56.916059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.915887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk" event={"ID":"f496dda2-72a5-4017-8d58-ac771958f301","Type":"ContainerDied","Data":"66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74"}
Sep 19 12:40:56.916059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:40:56.915936    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66edab04022e9da5b8aefde89ab4d0ae6668109a2df347b7b5a7d4a60f213d74"
Sep 19 12:41:04.918227 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:04.918191    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:41:04.918813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:04.918447    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" podUID="feabcbeb-282b-4c43-83b7-5f4368508edb" containerName="payload-processing" containerID="cri-o://d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5" gracePeriod=30
Sep 19 12:41:04.922300 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:04.922264    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:41:04.922611 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:04.922520    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" podUID="93d5f16a-899f-4fcb-b949-993c0cc48e73" containerName="payload-pre-processing" containerID="cri-o://54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573" gracePeriod=30
Sep 19 12:41:05.095466 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.095417    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"]
Sep 19 12:41:05.098483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.098456    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-21b2975e-mkggk"]
Sep 19 12:41:05.208484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.208457    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:41:05.212620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.212597    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:41:05.275657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.275627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume\") pod \"feabcbeb-282b-4c43-83b7-5f4368508edb\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") "
Sep 19 12:41:05.275844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.275673    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l62ld\" (UniqueName: \"kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld\") pod \"93d5f16a-899f-4fcb-b949-993c0cc48e73\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") "
Sep 19 12:41:05.275844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.275742    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs2k2\" (UniqueName: \"kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2\") pod \"feabcbeb-282b-4c43-83b7-5f4368508edb\" (UID: \"feabcbeb-282b-4c43-83b7-5f4368508edb\") "
Sep 19 12:41:05.275844 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.275796    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume\") pod \"93d5f16a-899f-4fcb-b949-993c0cc48e73\" (UID: \"93d5f16a-899f-4fcb-b949-993c0cc48e73\") "
Sep 19 12:41:05.276094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.276070    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "feabcbeb-282b-4c43-83b7-5f4368508edb" (UID: "feabcbeb-282b-4c43-83b7-5f4368508edb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:41:05.276237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.276213    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93d5f16a-899f-4fcb-b949-993c0cc48e73" (UID: "93d5f16a-899f-4fcb-b949-993c0cc48e73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:41:05.277904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.277884    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld" (OuterVolumeSpecName: "kube-api-access-l62ld") pod "93d5f16a-899f-4fcb-b949-993c0cc48e73" (UID: "93d5f16a-899f-4fcb-b949-993c0cc48e73"). InnerVolumeSpecName "kube-api-access-l62ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:41:05.277972 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.277911    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2" (OuterVolumeSpecName: "kube-api-access-qs2k2") pod "feabcbeb-282b-4c43-83b7-5f4368508edb" (UID: "feabcbeb-282b-4c43-83b7-5f4368508edb"). InnerVolumeSpecName "kube-api-access-qs2k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:41:05.376785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.376743    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feabcbeb-282b-4c43-83b7-5f4368508edb-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:41:05.376785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.376776    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l62ld\" (UniqueName: \"kubernetes.io/projected/93d5f16a-899f-4fcb-b949-993c0cc48e73-kube-api-access-l62ld\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:41:05.376785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.376785    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs2k2\" (UniqueName: \"kubernetes.io/projected/feabcbeb-282b-4c43-83b7-5f4368508edb-kube-api-access-qs2k2\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:41:05.376785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.376794    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d5f16a-899f-4fcb-b949-993c0cc48e73-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:41:05.722353 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.722320    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f496dda2-72a5-4017-8d58-ac771958f301" path="/var/lib/kubelet/pods/f496dda2-72a5-4017-8d58-ac771958f301/volumes"
Sep 19 12:41:05.965606 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.965573    2568 generic.go:356] "Generic (PLEG): container finished" podID="93d5f16a-899f-4fcb-b949-993c0cc48e73" containerID="54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573" exitCode=0
Sep 19 12:41:05.965606 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.965613    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"
Sep 19 12:41:05.966139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.965662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" event={"ID":"93d5f16a-899f-4fcb-b949-993c0cc48e73","Type":"ContainerDied","Data":"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"}
Sep 19 12:41:05.966139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.965713    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx" event={"ID":"93d5f16a-899f-4fcb-b949-993c0cc48e73","Type":"ContainerDied","Data":"71be4571e8a4c6634df120448a5be5e71fe1d71e490f25572acb250d16e70de0"}
Sep 19 12:41:05.966139 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.965735    2568 scope.go:117] "RemoveContainer" containerID="54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"
Sep 19 12:41:05.967267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.967244    2568 generic.go:356] "Generic (PLEG): container finished" podID="feabcbeb-282b-4c43-83b7-5f4368508edb" containerID="d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5" exitCode=0
Sep 19 12:41:05.967377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.967273    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" event={"ID":"feabcbeb-282b-4c43-83b7-5f4368508edb","Type":"ContainerDied","Data":"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"}
Sep 19 12:41:05.967377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.967303    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"
Sep 19 12:41:05.967377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.967325    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj" event={"ID":"feabcbeb-282b-4c43-83b7-5f4368508edb","Type":"ContainerDied","Data":"7f4fb104ce44102dfd825a7bee4d66535b50335abe2e768a1e4a02993ab3d51e"}
Sep 19 12:41:05.974959 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.974944    2568 scope.go:117] "RemoveContainer" containerID="54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"
Sep 19 12:41:05.975207 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:41:05.975189    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573\": container with ID starting with 54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573 not found: ID does not exist" containerID="54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"
Sep 19 12:41:05.975251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.975217    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573"} err="failed to get container status \"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573\": rpc error: code = NotFound desc = could not find container \"54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573\": container with ID starting with 54ebab6ad4b491b93676d821b93bd56e87eeb4d1a433b0091641f0eddf3e4573 not found: ID does not exist"
Sep 19 12:41:05.975251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.975235    2568 scope.go:117] "RemoveContainer" containerID="d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"
Sep 19 12:41:05.983129 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.983112    2568 scope.go:117] "RemoveContainer" containerID="d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"
Sep 19 12:41:05.983418 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:41:05.983397    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5\": container with ID starting with d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5 not found: ID does not exist" containerID="d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"
Sep 19 12:41:05.983527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.983423    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5"} err="failed to get container status \"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5\": rpc error: code = NotFound desc = could not find container \"d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5\": container with ID starting with d1174677e24cece529ad1cd6c83a7d96018c474a883e9f7fb2553490cde870b5 not found: ID does not exist"
Sep 19 12:41:05.988643 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.988623    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:41:05.995391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:05.995369    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-21b2975e-64b5fb7466-cxxhx"]
Sep 19 12:41:06.009212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.009185    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:41:06.013512 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.013471    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-21b2975e-656b544f6f-fqtxj"]
Sep 19 12:41:06.161951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.161923    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:41:06.162319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162307    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feabcbeb-282b-4c43-83b7-5f4368508edb" containerName="payload-processing"
Sep 19 12:41:06.162362 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162321    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="feabcbeb-282b-4c43-83b7-5f4368508edb" containerName="payload-processing"
Sep 19 12:41:06.162362 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162336    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f496dda2-72a5-4017-8d58-ac771958f301" containerName="revoke-keys"
Sep 19 12:41:06.162362 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162341    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f496dda2-72a5-4017-8d58-ac771958f301" containerName="revoke-keys"
Sep 19 12:41:06.162362 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162349    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d5f16a-899f-4fcb-b949-993c0cc48e73" containerName="payload-pre-processing"
Sep 19 12:41:06.162362 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162354    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d5f16a-899f-4fcb-b949-993c0cc48e73" containerName="payload-pre-processing"
Sep 19 12:41:06.162527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162408    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="feabcbeb-282b-4c43-83b7-5f4368508edb" containerName="payload-processing"
Sep 19 12:41:06.162527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162416    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d5f16a-899f-4fcb-b949-993c0cc48e73" containerName="payload-pre-processing"
Sep 19 12:41:06.162527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.162422    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f496dda2-72a5-4017-8d58-ac771958f301" containerName="revoke-keys"
Sep 19 12:41:06.166620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.166602    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.168911 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.168895    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-b4e91c-openshift-default-dockercfg-7mpnn\""
Sep 19 12:41:06.180175 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.180152    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:41:06.286445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286353    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286390    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286411    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286597    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286636    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286672    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286696    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.286909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.286786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9xkb\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.387986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.387959    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.387986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.387991    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388006    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388026    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388055    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9xkb\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388127    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388153    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388179    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388412    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388552    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388573    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.388741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.388662    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.389083 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.389062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.390539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.390519    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.390609 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.390592    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.395856 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.395832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.396434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.396414    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9xkb\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb\") pod \"e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") " pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.476700 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.476654    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:06.754738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.754716    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:41:06.757316 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:41:06.757292    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6399c1a_1755_46ea_be8d_ae742ea69995.slice/crio-399c083648469a46907bfcf41ed76c140b56867a97c1ecf9174499fda6c3aa0a WatchSource:0}: Error finding container 399c083648469a46907bfcf41ed76c140b56867a97c1ecf9174499fda6c3aa0a: Status 404 returned error can't find the container with id 399c083648469a46907bfcf41ed76c140b56867a97c1ecf9174499fda6c3aa0a
Sep 19 12:41:06.759537 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.759483    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:06.759594 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.759574    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:06.759630 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.759600    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:06.977249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.977212    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerStarted","Data":"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"}
Sep 19 12:41:06.977740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.977277    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerStarted","Data":"399c083648469a46907bfcf41ed76c140b56867a97c1ecf9174499fda6c3aa0a"}
Sep 19 12:41:06.996998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:06.996944    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podStartSLOduration=0.996928273 podStartE2EDuration="996.928273ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:41:06.995086786 +0000 UTC m=+951.284642554" watchObservedRunningTime="2026-09-19 12:41:06.996928273 +0000 UTC m=+951.286484040"
Sep 19 12:41:07.477126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.477093    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:07.482649 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.482623    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:07.721487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.721447    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d5f16a-899f-4fcb-b949-993c0cc48e73" path="/var/lib/kubelet/pods/93d5f16a-899f-4fcb-b949-993c0cc48e73/volumes"
Sep 19 12:41:07.721927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.721907    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feabcbeb-282b-4c43-83b7-5f4368508edb" path="/var/lib/kubelet/pods/feabcbeb-282b-4c43-83b7-5f4368508edb/volumes"
Sep 19 12:41:07.982296 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.982271    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:07.983293 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:07.983271    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:12.808827 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.808793    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:41:12.813666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.813648    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:12.817600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.817576    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-b4e91c\""
Sep 19 12:41:12.817719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.817630    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-b4e91c-dockercfg-xgwg9\""
Sep 19 12:41:12.832040 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.832017    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:41:12.844777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.844753    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:41:12.848213 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.848197    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:12.857541 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.857519    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:41:12.948254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.948220    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwfjf\" (UniqueName: \"kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:12.948438 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.948265    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:12.948438 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.948355    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stz74\" (UniqueName: \"kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:12.948438 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:12.948393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.049941 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.049908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwfjf\" (UniqueName: \"kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:13.050076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.049946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:13.050076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.050010    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stz74\" (UniqueName: \"kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.050076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.050040    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.050770 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.050745    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:13.050885 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.050755    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.057669 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.057638    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stz74\" (UniqueName: \"kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74\") pod \"payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.057900 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.057885    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwfjf\" (UniqueName: \"kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf\") pod \"payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") " pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:13.124098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.124033    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:13.159197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.159168    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:13.394744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.394720    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:41:13.396744 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:41:13.396709    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8497a068_0452_4dc9_8d33_9e5443c9e08b.slice/crio-a4c80a744e2f5b08aadc4bd069fb931e20f9a9ef88aaf978d5908d1c5d1093c8 WatchSource:0}: Error finding container a4c80a744e2f5b08aadc4bd069fb931e20f9a9ef88aaf978d5908d1c5d1093c8: Status 404 returned error can't find the container with id a4c80a744e2f5b08aadc4bd069fb931e20f9a9ef88aaf978d5908d1c5d1093c8
Sep 19 12:41:13.439753 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:13.439733    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:41:13.451654 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:41:13.451631    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37f43a8_d7e3_45e6_bc9d_55c95187409d.slice/crio-246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2 WatchSource:0}: Error finding container 246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2: Status 404 returned error can't find the container with id 246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2
Sep 19 12:41:14.019019 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.018987    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" event={"ID":"8497a068-0452-4dc9-8d33-9e5443c9e08b","Type":"ContainerStarted","Data":"b43326a70daa91a4f8d7b5314501b140203a13b80b60976a11120b03af39a1ef"}
Sep 19 12:41:14.019474 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.019044    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" event={"ID":"8497a068-0452-4dc9-8d33-9e5443c9e08b","Type":"ContainerStarted","Data":"a4c80a744e2f5b08aadc4bd069fb931e20f9a9ef88aaf978d5908d1c5d1093c8"}
Sep 19 12:41:14.019474 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.019073    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:14.020740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.020716    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" event={"ID":"e37f43a8-d7e3-45e6-bc9d-55c95187409d","Type":"ContainerStarted","Data":"d7b45aa094f250dbe1130ecc6d3ab2bd4a9e707a3f4be7984197c808793130fb"}
Sep 19 12:41:14.020860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.020759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" event={"ID":"e37f43a8-d7e3-45e6-bc9d-55c95187409d","Type":"ContainerStarted","Data":"246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2"}
Sep 19 12:41:14.020860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.020780    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:14.035646 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.035606    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" podStartSLOduration=2.035591463 podStartE2EDuration="2.035591463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:41:14.03457113 +0000 UTC m=+958.324126900" watchObservedRunningTime="2026-09-19 12:41:14.035591463 +0000 UTC m=+958.325147230"
Sep 19 12:41:14.064887 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:14.064836    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" podStartSLOduration=2.064818083 podStartE2EDuration="2.064818083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:41:14.062125599 +0000 UTC m=+958.351681369" watchObservedRunningTime="2026-09-19 12:41:14.064818083 +0000 UTC m=+958.354373854"
Sep 19 12:41:25.030895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:25.030863    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:41:25.031343 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:25.030935    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:41:33.139136 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.139103    2568 generic.go:356] "Generic (PLEG): container finished" podID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerID="65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c" exitCode=0
Sep 19 12:41:33.139573 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.139178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerDied","Data":"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"}
Sep 19 12:41:33.139624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.139612    2568 scope.go:117] "RemoveContainer" containerID="65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"
Sep 19 12:41:33.146851 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.141368    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:33.146851 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.141698    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:33.146851 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:33.141852    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:41:34.145600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:34.145567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerStarted","Data":"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"}
Sep 19 12:41:34.478580 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:34.478470    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:34.478931 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:34.478906    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 19 12:41:34.479113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:34.479089    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 19 12:41:35.478377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:35.477947    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 19 12:41:35.478377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:35.478009    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 19 12:41:36.477723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:36.477683    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 19 12:41:36.478020 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:36.477990    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 19 12:41:37.478056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:37.478008    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 19 12:41:37.478575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:37.478079    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 19 12:41:37.982278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:37.982244    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:38.477696 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:38.477649    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 19 12:41:38.477908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:38.477730    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 19 12:41:39.481391 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:39.481363    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:41:39.482284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:41:39.482264    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:42:40.099849 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.099810    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:42:40.109224 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.109196    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.112306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.112279    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f9a7df96-openshift-default-dockercfg-k4qcc\""
Sep 19 12:42:40.112608 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.112297    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:42:40.117870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.117841    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.117997 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.117938    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.117997 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.117991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118103 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118024    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118154 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118131    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118198 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw9mr\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118238 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118220    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118277 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118246    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.118321 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.118287    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.218834 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.218834 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218844    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218866    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218939    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw9mr\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218968    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.218991    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219019    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219094 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219280    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219395    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219609 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219588    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.219751 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.219722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.220188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.220162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.221691 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.221663    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.221900 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.221883    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.227121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.227097    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw9mr\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.227555 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.227418    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token\") pod \"e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") " pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.422111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.422073    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:40.709877 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.709852    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:42:40.711536 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:42:40.711512    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d9d591_9c2a_4e7a_9a29_b692a5972612.slice/crio-c3a7205c8c009234bd0b6906c7e58f89fbbde5f55e932fd4c9891d380d7b4fd8 WatchSource:0}: Error finding container c3a7205c8c009234bd0b6906c7e58f89fbbde5f55e932fd4c9891d380d7b4fd8: Status 404 returned error can't find the container with id c3a7205c8c009234bd0b6906c7e58f89fbbde5f55e932fd4c9891d380d7b4fd8
Sep 19 12:42:40.713870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.713839    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:42:40.713980 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.713913    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:42:40.713980 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:40.713957    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:42:41.490147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:41.490112    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerStarted","Data":"524823c18dfbe9eb4c246ddc6cae97f4cb90437e470396576a1455fc4e66bc8b"}
Sep 19 12:42:41.490576 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:41.490169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerStarted","Data":"c3a7205c8c009234bd0b6906c7e58f89fbbde5f55e932fd4c9891d380d7b4fd8"}
Sep 19 12:42:41.512444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:41.512394    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podStartSLOduration=1.512376712 podStartE2EDuration="1.512376712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:42:41.511328654 +0000 UTC m=+1045.800884424" watchObservedRunningTime="2026-09-19 12:42:41.512376712 +0000 UTC m=+1045.801932480"
Sep 19 12:42:42.422728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:42.422696    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:42.428036 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:42.428010    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:42.495421 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:42.495392    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:42:42.496269 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:42:42.496251    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:13.536693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:13.536652    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 12:43:13.537258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:13.536716    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 12:43:14.660432 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.660401    2568 generic.go:356] "Generic (PLEG): container finished" podID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerID="524823c18dfbe9eb4c246ddc6cae97f4cb90437e470396576a1455fc4e66bc8b" exitCode=0
Sep 19 12:43:14.660964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.660445    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerDied","Data":"524823c18dfbe9eb4c246ddc6cae97f4cb90437e470396576a1455fc4e66bc8b"}
Sep 19 12:43:14.660964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.660835    2568 scope.go:117] "RemoveContainer" containerID="524823c18dfbe9eb4c246ddc6cae97f4cb90437e470396576a1455fc4e66bc8b"
Sep 19 12:43:14.662177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.662146    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:14.662284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.662215    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:14.662284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:14.662251    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:15.671268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:15.671218    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerStarted","Data":"1f608af63ce316046b1e36158ae1e3393581bbb6889a865519f95654f599e340"}
Sep 19 12:43:16.422749 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:16.422711    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:16.423007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:16.422984    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 19 12:43:16.423055 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:16.423033    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 19 12:43:17.422904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:17.422871    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 19 12:43:17.423303 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:17.422934    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 19 12:43:18.422917 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:18.422877    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 19 12:43:18.423400 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:18.422939    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 19 12:43:19.422925 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:19.422886    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 19 12:43:19.423324 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:19.422937    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 19 12:43:20.422686 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:20.422648    2568 patch_prober.go:28] interesting pod/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 19 12:43:20.422870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:20.422706    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 19 12:43:21.426279 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:21.426245    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:21.426715 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:21.426529    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:21.427244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:21.427228    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:22.654341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.654308    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"]
Sep 19 12:43:22.657106 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.657089    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:22.659587 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.659562    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:43:22.659721 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.659600    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:43:22.666120 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.666089    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"]
Sep 19 12:43:22.835867 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.835829    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b\" (UID: \"ec80d612-8792-4ff0-bf8e-70797c36073e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:22.936896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.936808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b\" (UID: \"ec80d612-8792-4ff0-bf8e-70797c36073e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:22.937488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.937461    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b\" (UID: \"ec80d612-8792-4ff0-bf8e-70797c36073e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:22.968451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:22.968422    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:23.238676 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:23.238647    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"]
Sep 19 12:43:23.241160 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:43:23.241135    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec80d612_8792_4ff0_bf8e_70797c36073e.slice/crio-0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548 WatchSource:0}: Error finding container 0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548: Status 404 returned error can't find the container with id 0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548
Sep 19 12:43:23.729554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:23.729519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b" event={"ID":"ec80d612-8792-4ff0-bf8e-70797c36073e","Type":"ContainerStarted","Data":"1462ff5f6509edeaab3d4eaded9ebc966af1e946d9dba2453b9780aed91e9fc0"}
Sep 19 12:43:23.729923 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:23.729567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b" event={"ID":"ec80d612-8792-4ff0-bf8e-70797c36073e","Type":"ContainerStarted","Data":"0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548"}
Sep 19 12:43:23.744873 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:23.744827    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b" podStartSLOduration=1.74481316 podStartE2EDuration="1.74481316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:43:23.743552387 +0000 UTC m=+1088.033108153" watchObservedRunningTime="2026-09-19 12:43:23.74481316 +0000 UTC m=+1088.034368929"
Sep 19 12:43:24.735461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:24.735428    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec80d612-8792-4ff0-bf8e-70797c36073e" containerID="1462ff5f6509edeaab3d4eaded9ebc966af1e946d9dba2453b9780aed91e9fc0" exitCode=0
Sep 19 12:43:24.735881 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:24.735530    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b" event={"ID":"ec80d612-8792-4ff0-bf8e-70797c36073e","Type":"ContainerDied","Data":"1462ff5f6509edeaab3d4eaded9ebc966af1e946d9dba2453b9780aed91e9fc0"}
Sep 19 12:43:25.905456 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:25.905427    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:25.966474 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:25.966443    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca\") pod \"ec80d612-8792-4ff0-bf8e-70797c36073e\" (UID: \"ec80d612-8792-4ff0-bf8e-70797c36073e\") "
Sep 19 12:43:25.966846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:25.966823    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ec80d612-8792-4ff0-bf8e-70797c36073e" (UID: "ec80d612-8792-4ff0-bf8e-70797c36073e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:26.067728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:26.067629    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec80d612-8792-4ff0-bf8e-70797c36073e-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:26.748419 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:26.748392    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"
Sep 19 12:43:26.748631 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:26.748435    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b" event={"ID":"ec80d612-8792-4ff0-bf8e-70797c36073e","Type":"ContainerDied","Data":"0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548"}
Sep 19 12:43:26.748631 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:26.748480    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fe2d379bbd38a41cacf2fbdb5c6c8e53e41819e488f3e8850228d16ca80e548"
Sep 19 12:43:34.197990 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:34.197918    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"]
Sep 19 12:43:34.199991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:34.199970    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f9a7df96-hc62b"]
Sep 19 12:43:35.722918 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:35.722883    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec80d612-8792-4ff0-bf8e-70797c36073e" path="/var/lib/kubelet/pods/ec80d612-8792-4ff0-bf8e-70797c36073e/volumes"
Sep 19 12:43:40.642299 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:40.642265    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:43:40.642888 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:40.642553    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy" containerID="cri-o://1f608af63ce316046b1e36158ae1e3393581bbb6889a865519f95654f599e340" gracePeriod=30
Sep 19 12:43:42.068984 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.068945    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"]
Sep 19 12:43:42.069420 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.069376    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec80d612-8792-4ff0-bf8e-70797c36073e" containerName="revoke-keys"
Sep 19 12:43:42.069420 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.069389    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec80d612-8792-4ff0-bf8e-70797c36073e" containerName="revoke-keys"
Sep 19 12:43:42.069533 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.069465    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec80d612-8792-4ff0-bf8e-70797c36073e" containerName="revoke-keys"
Sep 19 12:43:42.072592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.072575    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:42.074912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.074892    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:43:42.074912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.074901    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:43:42.079871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.079846    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"]
Sep 19 12:43:42.120951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.120924    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b4e91c-htf79\" (UID: \"a0a9b614-13de-48da-9802-dad6b722f40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:42.221744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.221711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b4e91c-htf79\" (UID: \"a0a9b614-13de-48da-9802-dad6b722f40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:42.222276 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.222257    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b4e91c-htf79\" (UID: \"a0a9b614-13de-48da-9802-dad6b722f40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:42.384176 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.384141    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:42.658434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.658410    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"]
Sep 19 12:43:42.660562 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:43:42.660533    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a9b614_13de_48da_9802_dad6b722f40f.slice/crio-24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843 WatchSource:0}: Error finding container 24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843: Status 404 returned error can't find the container with id 24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843
Sep 19 12:43:42.832856 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:42.832811    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79" event={"ID":"a0a9b614-13de-48da-9802-dad6b722f40f","Type":"ContainerStarted","Data":"24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843"}
Sep 19 12:43:43.839651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:43.839620    2568 generic.go:356] "Generic (PLEG): container finished" podID="a0a9b614-13de-48da-9802-dad6b722f40f" containerID="09ed97316a867b5e7d02406645b6661669445c5ca2c0d0666b5e67113e3cd1b3" exitCode=0
Sep 19 12:43:43.840057 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:43.839673    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79" event={"ID":"a0a9b614-13de-48da-9802-dad6b722f40f","Type":"ContainerDied","Data":"09ed97316a867b5e7d02406645b6661669445c5ca2c0d0666b5e67113e3cd1b3"}
Sep 19 12:43:44.993284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:44.993259    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:45.049113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.049084    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca\") pod \"a0a9b614-13de-48da-9802-dad6b722f40f\" (UID: \"a0a9b614-13de-48da-9802-dad6b722f40f\") "
Sep 19 12:43:45.049448 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.049423    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0a9b614-13de-48da-9802-dad6b722f40f" (UID: "a0a9b614-13de-48da-9802-dad6b722f40f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:45.150296 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.150266    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a9b614-13de-48da-9802-dad6b722f40f-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.852057 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.852019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79" event={"ID":"a0a9b614-13de-48da-9802-dad6b722f40f","Type":"ContainerDied","Data":"24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843"}
Sep 19 12:43:45.852222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.852083    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24dc64a2bce21bb7f49a820b313f26c3cb8fc4285823adcd4de62cb871992843"
Sep 19 12:43:45.852222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.852032    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"
Sep 19 12:43:45.854227 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.854195    2568 generic.go:356] "Generic (PLEG): container finished" podID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerID="1f608af63ce316046b1e36158ae1e3393581bbb6889a865519f95654f599e340" exitCode=0
Sep 19 12:43:45.854372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.854224    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerDied","Data":"1f608af63ce316046b1e36158ae1e3393581bbb6889a865519f95654f599e340"}
Sep 19 12:43:45.854372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.854284    2568 scope.go:117] "RemoveContainer" containerID="524823c18dfbe9eb4c246ddc6cae97f4cb90437e470396576a1455fc4e66bc8b"
Sep 19 12:43:45.894710 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.894691    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:45.958790 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.958706    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.958790 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.958748    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.958872    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.958914    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959007    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw9mr\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959048    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:45.959119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959061    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959108    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959151    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959180    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo\") pod \"62d9d591-9c2a-4e7a-9a29-b692a5972612\" (UID: \"62d9d591-9c2a-4e7a-9a29-b692a5972612\") "
Sep 19 12:43:45.959325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959212    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:43:45.959482 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959391    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data" (OuterVolumeSpecName: "istio-data") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:43:45.959482 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959426    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:43:45.959482 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959434    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:43:45.959677 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959596    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.959677 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959618    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d9d591-9c2a-4e7a-9a29-b692a5972612-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.959677 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959632    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.959677 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959646    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.959677 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.959658    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:45.960965 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.960942    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token" (OuterVolumeSpecName: "istio-token") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:45.961065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.961025    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:43:45.961115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.961097    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr" (OuterVolumeSpecName: "kube-api-access-sw9mr") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "kube-api-access-sw9mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:45.961246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:45.961228    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62d9d591-9c2a-4e7a-9a29-b692a5972612" (UID: "62d9d591-9c2a-4e7a-9a29-b692a5972612"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:43:46.060634 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.060602    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:46.060634 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.060627    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:46.060634 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.060640    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:46.061085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.060648    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw9mr\" (UniqueName: \"kubernetes.io/projected/62d9d591-9c2a-4e7a-9a29-b692a5972612-kube-api-access-sw9mr\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:46.860551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.860519    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"
Sep 19 12:43:46.860849 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.860561    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8" event={"ID":"62d9d591-9c2a-4e7a-9a29-b692a5972612","Type":"ContainerDied","Data":"c3a7205c8c009234bd0b6906c7e58f89fbbde5f55e932fd4c9891d380d7b4fd8"}
Sep 19 12:43:46.860849 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.860628    2568 scope.go:117] "RemoveContainer" containerID="1f608af63ce316046b1e36158ae1e3393581bbb6889a865519f95654f599e340"
Sep 19 12:43:46.881119 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.881088    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:43:46.885116 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:46.885086    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f9a7df96-openshift-default-6fc96b9c5b-qtsn8"]
Sep 19 12:43:47.722302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:47.722257    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" path="/var/lib/kubelet/pods/62d9d591-9c2a-4e7a-9a29-b692a5972612/volumes"
Sep 19 12:43:52.610729 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.610697    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:43:52.611209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.610960    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" podUID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" containerName="payload-processing" containerID="cri-o://d7b45aa094f250dbe1130ecc6d3ab2bd4a9e707a3f4be7984197c808793130fb" gracePeriod=30
Sep 19 12:43:52.618275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.618250    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:43:52.618489 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.618464    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" podUID="8497a068-0452-4dc9-8d33-9e5443c9e08b" containerName="payload-pre-processing" containerID="cri-o://b43326a70daa91a4f8d7b5314501b140203a13b80b60976a11120b03af39a1ef" gracePeriod=30
Sep 19 12:43:52.769748 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.769701    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"]
Sep 19 12:43:52.771485 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.771459    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b4e91c-htf79"]
Sep 19 12:43:52.902857 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.902824    2568 generic.go:356] "Generic (PLEG): container finished" podID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" containerID="d7b45aa094f250dbe1130ecc6d3ab2bd4a9e707a3f4be7984197c808793130fb" exitCode=0
Sep 19 12:43:52.903416 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.903383    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" event={"ID":"e37f43a8-d7e3-45e6-bc9d-55c95187409d","Type":"ContainerDied","Data":"d7b45aa094f250dbe1130ecc6d3ab2bd4a9e707a3f4be7984197c808793130fb"}
Sep 19 12:43:52.903596 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.903451    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2" event={"ID":"e37f43a8-d7e3-45e6-bc9d-55c95187409d","Type":"ContainerDied","Data":"246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2"}
Sep 19 12:43:52.903596 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.903477    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="246719e1d6a3372aaaa303989c5dad53464c23c551f9a533b4b4d361b6026fa2"
Sep 19 12:43:52.905028 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.905006    2568 generic.go:356] "Generic (PLEG): container finished" podID="8497a068-0452-4dc9-8d33-9e5443c9e08b" containerID="b43326a70daa91a4f8d7b5314501b140203a13b80b60976a11120b03af39a1ef" exitCode=0
Sep 19 12:43:52.905144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.905053    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" event={"ID":"8497a068-0452-4dc9-8d33-9e5443c9e08b","Type":"ContainerDied","Data":"b43326a70daa91a4f8d7b5314501b140203a13b80b60976a11120b03af39a1ef"}
Sep 19 12:43:52.923405 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.923384    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:43:52.926740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:52.926719    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:43:53.023337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023305    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwfjf\" (UniqueName: \"kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf\") pod \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") "
Sep 19 12:43:53.023568 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023355    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stz74\" (UniqueName: \"kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74\") pod \"8497a068-0452-4dc9-8d33-9e5443c9e08b\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") "
Sep 19 12:43:53.023568 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023396    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume\") pod \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\" (UID: \"e37f43a8-d7e3-45e6-bc9d-55c95187409d\") "
Sep 19 12:43:53.023568 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume\") pod \"8497a068-0452-4dc9-8d33-9e5443c9e08b\" (UID: \"8497a068-0452-4dc9-8d33-9e5443c9e08b\") "
Sep 19 12:43:53.023901 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023874    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e37f43a8-d7e3-45e6-bc9d-55c95187409d" (UID: "e37f43a8-d7e3-45e6-bc9d-55c95187409d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:53.023950 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.023875    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8497a068-0452-4dc9-8d33-9e5443c9e08b" (UID: "8497a068-0452-4dc9-8d33-9e5443c9e08b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:53.025642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.025616    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf" (OuterVolumeSpecName: "kube-api-access-zwfjf") pod "e37f43a8-d7e3-45e6-bc9d-55c95187409d" (UID: "e37f43a8-d7e3-45e6-bc9d-55c95187409d"). InnerVolumeSpecName "kube-api-access-zwfjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:53.025722 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.025702    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74" (OuterVolumeSpecName: "kube-api-access-stz74") pod "8497a068-0452-4dc9-8d33-9e5443c9e08b" (UID: "8497a068-0452-4dc9-8d33-9e5443c9e08b"). InnerVolumeSpecName "kube-api-access-stz74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:53.125166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.125127    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwfjf\" (UniqueName: \"kubernetes.io/projected/e37f43a8-d7e3-45e6-bc9d-55c95187409d-kube-api-access-zwfjf\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.125166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.125161    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stz74\" (UniqueName: \"kubernetes.io/projected/8497a068-0452-4dc9-8d33-9e5443c9e08b-kube-api-access-stz74\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.125166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.125171    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e37f43a8-d7e3-45e6-bc9d-55c95187409d-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.125399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.125180    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497a068-0452-4dc9-8d33-9e5443c9e08b-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.721427 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.721396    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a9b614-13de-48da-9802-dad6b722f40f" path="/var/lib/kubelet/pods/a0a9b614-13de-48da-9802-dad6b722f40f/volumes"
Sep 19 12:43:53.911567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.911464    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"
Sep 19 12:43:53.911567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.911508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6" event={"ID":"8497a068-0452-4dc9-8d33-9e5443c9e08b","Type":"ContainerDied","Data":"a4c80a744e2f5b08aadc4bd069fb931e20f9a9ef88aaf978d5908d1c5d1093c8"}
Sep 19 12:43:53.911567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.911566    2568 scope.go:117] "RemoveContainer" containerID="b43326a70daa91a4f8d7b5314501b140203a13b80b60976a11120b03af39a1ef"
Sep 19 12:43:53.911812 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.911747    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"
Sep 19 12:43:53.932548 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.932521    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:43:53.939150 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.939128    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b4e91c-5d957cb5c4-z7qw6"]
Sep 19 12:43:53.951160 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.951130    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:43:53.953009 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:53.952990    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b4e91c-5d7bfd87fc-7z8q2"]
Sep 19 12:43:54.624402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.624370    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:43:54.624949 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.624929    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8497a068-0452-4dc9-8d33-9e5443c9e08b" containerName="payload-pre-processing"
Sep 19 12:43:54.625004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.624954    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8497a068-0452-4dc9-8d33-9e5443c9e08b" containerName="payload-pre-processing"
Sep 19 12:43:54.625004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.624975    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a9b614-13de-48da-9802-dad6b722f40f" containerName="revoke-keys"
Sep 19 12:43:54.625004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.624983    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a9b614-13de-48da-9802-dad6b722f40f" containerName="revoke-keys"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625010    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625018    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625038    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625047    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625064    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" containerName="payload-processing"
Sep 19 12:43:54.625098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625073    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" containerName="payload-processing"
Sep 19 12:43:54.625327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625158    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8497a068-0452-4dc9-8d33-9e5443c9e08b" containerName="payload-pre-processing"
Sep 19 12:43:54.625327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625171    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" containerName="payload-processing"
Sep 19 12:43:54.625327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625179    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.625327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625189    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a9b614-13de-48da-9802-dad6b722f40f" containerName="revoke-keys"
Sep 19 12:43:54.625327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.625198    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d9d591-9c2a-4e7a-9a29-b692a5972612" containerName="istio-proxy"
Sep 19 12:43:54.629731 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.629707    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.632057 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.632036    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-2a487c-openshift-default-dockercfg-ddmx6\""
Sep 19 12:43:54.639958 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.639935    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:43:54.742576 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.742527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.742822    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.742868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.742922    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.742971    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.743004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.743039    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.743077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.743394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.743134    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k7nx\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844466 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844434    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844658 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k7nx\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844755 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844732    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844767    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844864    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844895    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.844939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.844927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.845269 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.845247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.845350 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.845281    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.845350 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.845330    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.845469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.845442    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.845469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.845449    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.847092 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.847064    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.847207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.847125    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.855916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.855893    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k7nx\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.856024 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.855898    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token\") pod \"e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") " pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:54.942629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:54.942604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:55.218411 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.218321    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:43:55.221601 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:43:55.221563    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308d75b3_ebd4_45c3_96b2_047a11e12b6c.slice/crio-3b78b1c735e92294b7b0c66753df934b8ed09f64b419c7495daa596b1d07a2db WatchSource:0}: Error finding container 3b78b1c735e92294b7b0c66753df934b8ed09f64b419c7495daa596b1d07a2db: Status 404 returned error can't find the container with id 3b78b1c735e92294b7b0c66753df934b8ed09f64b419c7495daa596b1d07a2db
Sep 19 12:43:55.223816 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.223781    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:55.223940 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.223848    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:55.223940 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.223873    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:43:55.723685 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.723644    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8497a068-0452-4dc9-8d33-9e5443c9e08b" path="/var/lib/kubelet/pods/8497a068-0452-4dc9-8d33-9e5443c9e08b/volumes"
Sep 19 12:43:55.724118 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.724098    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e37f43a8-d7e3-45e6-bc9d-55c95187409d" path="/var/lib/kubelet/pods/e37f43a8-d7e3-45e6-bc9d-55c95187409d/volumes"
Sep 19 12:43:55.927912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.927876    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerStarted","Data":"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"}
Sep 19 12:43:55.928371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.927923    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerStarted","Data":"3b78b1c735e92294b7b0c66753df934b8ed09f64b419c7495daa596b1d07a2db"}
Sep 19 12:43:55.942732 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.942702    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:55.954647 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:55.954593    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podStartSLOduration=1.954572176 podStartE2EDuration="1.954572176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:43:55.951953497 +0000 UTC m=+1120.241509266" watchObservedRunningTime="2026-09-19 12:43:55.954572176 +0000 UTC m=+1120.244127947"
Sep 19 12:43:56.948970 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:56.948945    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:57.939557 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:57.939523    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:43:57.940449 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:43:57.940428    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:44:01.996456 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:01.996418    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:44:02.001190 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.001171    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.003649 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.003627    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2a487c\""
Sep 19 12:44:02.003776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.003635    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2a487c-dockercfg-b4h9s\""
Sep 19 12:44:02.009251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.009230    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:44:02.020210 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.020188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.020310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.020221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvlw5\" (UniqueName: \"kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.121208 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.121168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.122738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.122713    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlw5\" (UniqueName: \"kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.122986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.122604    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.132098 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.132052    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlw5\" (UniqueName: \"kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5\") pod \"payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.312944 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.312849    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.585444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.585418    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:44:02.587357 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:44:02.587329    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e40fae_5cd8_4d93_b53e_0b378b2eee1e.slice/crio-6734335c5ca9fe6a22cbab0a269f40508bbd4612ebbd7d6eb2c6d958b84d9df9 WatchSource:0}: Error finding container 6734335c5ca9fe6a22cbab0a269f40508bbd4612ebbd7d6eb2c6d958b84d9df9: Status 404 returned error can't find the container with id 6734335c5ca9fe6a22cbab0a269f40508bbd4612ebbd7d6eb2c6d958b84d9df9
Sep 19 12:44:02.971424 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.971386    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" event={"ID":"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e","Type":"ContainerStarted","Data":"f7783f19f117b5752fbbb3f4adda8fcdf1c3c5471a2d1fad37bfb5c04217eafc"}
Sep 19 12:44:02.971642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.971442    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" event={"ID":"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e","Type":"ContainerStarted","Data":"6734335c5ca9fe6a22cbab0a269f40508bbd4612ebbd7d6eb2c6d958b84d9df9"}
Sep 19 12:44:02.971642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.971478    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:02.988492 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:02.988439    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" podStartSLOduration=1.988422696 podStartE2EDuration="1.988422696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:02.985972044 +0000 UTC m=+1127.275527814" watchObservedRunningTime="2026-09-19 12:44:02.988422696 +0000 UTC m=+1127.277978510"
Sep 19 12:44:13.979771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:13.979738    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:44:28.456423 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.456386    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:44:28.476728 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.476697    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:44:28.477100 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.477016    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.489983 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.489945    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-5068d9-openshift-default-dockercfg-6d7cx\""
Sep 19 12:44:28.594022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.593983    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.594366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.594340    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.594617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.594595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.594785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.594768    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.594987 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.594970    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j78mn\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.595133 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.595117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.595267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.595252    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.595433 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.595410    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.598538 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.595761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.696771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.696731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j78mn\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.697046 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.697021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.697185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.697169    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.697322 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.697306    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.697512 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.697477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.698641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.698710    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.698765    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.698787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.698824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.699001    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.699126    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699645 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.699613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.699831 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.699810    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.700912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.700892    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.701222 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.701176    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.706046 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.706021    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.706264 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.706245    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j78mn\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn\") pod \"e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") " pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:28.802777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:28.802670    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:29.088431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:29.087926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:44:29.090644 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:44:29.090612    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea661d4_d732_4162_b95a_a94e0ff79ac9.slice/crio-c6d68846bbdd8b30c718fc8d7da8c14bb59d3f3c5c64525bc306d1fa7b6c3808 WatchSource:0}: Error finding container c6d68846bbdd8b30c718fc8d7da8c14bb59d3f3c5c64525bc306d1fa7b6c3808: Status 404 returned error can't find the container with id c6d68846bbdd8b30c718fc8d7da8c14bb59d3f3c5c64525bc306d1fa7b6c3808
Sep 19 12:44:29.093136 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:29.093097    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:29.093228 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:29.093184    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:29.093273 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:29.093225    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:29.108557 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:29.108526    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerStarted","Data":"c6d68846bbdd8b30c718fc8d7da8c14bb59d3f3c5c64525bc306d1fa7b6c3808"}
Sep 19 12:44:30.115720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:30.115629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerStarted","Data":"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"}
Sep 19 12:44:30.139772 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:30.139719    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podStartSLOduration=2.13969971 podStartE2EDuration="2.13969971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:30.137473798 +0000 UTC m=+1154.427029579" watchObservedRunningTime="2026-09-19 12:44:30.13969971 +0000 UTC m=+1154.429255459"
Sep 19 12:44:30.803556 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:30.803516    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:30.808781 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:30.808753    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:31.122871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:31.122775    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:31.123747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:31.123726    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:35.309008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:35.308925    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:44:35.310554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:35.309191    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" containerID="cri-o://9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9" gracePeriod=30
Sep 19 12:44:36.061719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.061679    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:44:36.065776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.065747    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.068557 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.068533    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-e8f833-openshift-default-dockercfg-w5g4j\""
Sep 19 12:44:36.072122 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072164    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072229    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072542 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jm9p\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072542 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072542 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.072542 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.072455    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.077568 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.077544    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:44:36.156977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.156928    2568 generic.go:356] "Generic (PLEG): container finished" podID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerID="b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48" exitCode=0
Sep 19 12:44:36.157458 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.157214    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerDied","Data":"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"}
Sep 19 12:44:36.158741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.158720    2568 scope.go:117] "RemoveContainer" containerID="b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"
Sep 19 12:44:36.164015 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.163980    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:36.164162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.164072    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:36.164162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.164123    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:36.174076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174039    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jm9p\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.174309 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.174431 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174415    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.174593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174568    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.174877 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174856    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.175009 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.174995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.175128 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175113    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.175241 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175217    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.175310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175222    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.175310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175294    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.176598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.176598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.175834    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.176598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.176312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.176813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.176798    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.180636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.180601    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.185138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.185106    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.194732 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.194668    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.208831 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.207046    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jm9p\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p\") pod \"e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") " pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.377852 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.377816    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:36.660916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.660835    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:44:36.663968 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:44:36.663935    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe172484_05d1_4e64_8f82_10e10a3faa05.slice/crio-50a2ec1bbd9a7e93ae384f660a2086322c8a25630cfaf52d55bf3099a8e802e3 WatchSource:0}: Error finding container 50a2ec1bbd9a7e93ae384f660a2086322c8a25630cfaf52d55bf3099a8e802e3: Status 404 returned error can't find the container with id 50a2ec1bbd9a7e93ae384f660a2086322c8a25630cfaf52d55bf3099a8e802e3
Sep 19 12:44:36.666392 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.666361    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:36.666462 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.666424    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:36.666462 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:36.666455    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:37.169273 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.168215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerStarted","Data":"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"}
Sep 19 12:44:37.173164 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.171446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" event={"ID":"fe172484-05d1-4e64-8f82-10e10a3faa05","Type":"ContainerStarted","Data":"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"}
Sep 19 12:44:37.173164 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.171525    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" event={"ID":"fe172484-05d1-4e64-8f82-10e10a3faa05","Type":"ContainerStarted","Data":"50a2ec1bbd9a7e93ae384f660a2086322c8a25630cfaf52d55bf3099a8e802e3"}
Sep 19 12:44:37.207018 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.206962    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" podStartSLOduration=1.206943709 podStartE2EDuration="1.206943709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:37.206455853 +0000 UTC m=+1161.496011623" watchObservedRunningTime="2026-09-19 12:44:37.206943709 +0000 UTC m=+1161.496499479"
Sep 19 12:44:37.379902 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.379865    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:37.388950 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.388916    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:37.943248 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.943220    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:44:37.943481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.943458    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 19 12:44:37.943609 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:37.943543    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 19 12:44:38.182030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:38.182003    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:38.186059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:38.186032    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:44:38.943701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:38.943648    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 19 12:44:38.944189 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:38.943736    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 19 12:44:39.485995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:39.485952    2568 patch_prober.go:28] interesting pod/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 12:44:39.486178 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:39.486037    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:44:39.943792 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:39.943751    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 19 12:44:39.944198 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:39.943823    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 19 12:44:40.612399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.612375    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:44:40.740637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740598    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740635    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740684    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740707    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740728    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740763    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9xkb\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740811    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740833    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.740905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740892    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket\") pod \"f6399c1a-1755-46ea-be8d-ae742ea69995\" (UID: \"f6399c1a-1755-46ea-be8d-ae742ea69995\") "
Sep 19 12:44:40.741179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.740946    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:44:40.741179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741013    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:44:40.741179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741150    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:44:40.741335 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741220    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:44:40.741335 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741224    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:44:40.741446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741332    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.741446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741351    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6399c1a-1755-46ea-be8d-ae742ea69995-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.741446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741360    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.741446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741369    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.741446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.741379    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.743185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.743157    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:44:40.743185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.743172    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:44:40.743539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.743515    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb" (OuterVolumeSpecName: "kube-api-access-k9xkb") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "kube-api-access-k9xkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:44:40.743539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.743522    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6399c1a-1755-46ea-be8d-ae742ea69995" (UID: "f6399c1a-1755-46ea-be8d-ae742ea69995"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:44:40.842478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.842392    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.842478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.842420    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.842478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.842430    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6399c1a-1755-46ea-be8d-ae742ea69995-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.842478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.842438    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9xkb\" (UniqueName: \"kubernetes.io/projected/f6399c1a-1755-46ea-be8d-ae742ea69995-kube-api-access-k9xkb\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:44:40.943083 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.943046    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 19 12:44:40.943259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:40.943100    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 19 12:44:41.199839 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.199806    2568 generic.go:356] "Generic (PLEG): container finished" podID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerID="9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9" exitCode=0
Sep 19 12:44:41.200268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.199852    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerDied","Data":"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"}
Sep 19 12:44:41.200268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.199862    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"
Sep 19 12:44:41.200268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.199896    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz" event={"ID":"f6399c1a-1755-46ea-be8d-ae742ea69995","Type":"ContainerDied","Data":"399c083648469a46907bfcf41ed76c140b56867a97c1ecf9174499fda6c3aa0a"}
Sep 19 12:44:41.200268 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.199917    2568 scope.go:117] "RemoveContainer" containerID="9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"
Sep 19 12:44:41.215755 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.215732    2568 scope.go:117] "RemoveContainer" containerID="65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"
Sep 19 12:44:41.225003 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.224981    2568 scope.go:117] "RemoveContainer" containerID="9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"
Sep 19 12:44:41.225325 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:44:41.225305    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9\": container with ID starting with 9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9 not found: ID does not exist" containerID="9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"
Sep 19 12:44:41.225402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.225334    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9"} err="failed to get container status \"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9\": rpc error: code = NotFound desc = could not find container \"9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9\": container with ID starting with 9a0a04c15c5329d991fea4e5afc1575238639efdf861d39688405f2609b8afc9 not found: ID does not exist"
Sep 19 12:44:41.225402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.225353    2568 scope.go:117] "RemoveContainer" containerID="65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"
Sep 19 12:44:41.225670 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:44:41.225646    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c\": container with ID starting with 65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c not found: ID does not exist" containerID="65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"
Sep 19 12:44:41.225720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.225681    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c"} err="failed to get container status \"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c\": rpc error: code = NotFound desc = could not find container \"65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c\": container with ID starting with 65547ba75d923285b14b9b43cdf9a01784f108d477ab58e9ead4949179400e1c not found: ID does not exist"
Sep 19 12:44:41.226581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.226564    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:44:41.230109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.230084    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-b4e91c-openshift-default-75f9cd9cb-pglsz"]
Sep 19 12:44:41.722969 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.722938    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" path="/var/lib/kubelet/pods/f6399c1a-1755-46ea-be8d-ae742ea69995/volumes"
Sep 19 12:44:41.943385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.943347    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 19 12:44:41.943559 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:41.943405    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 19 12:44:42.940221 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:42.940185    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:44:42.947134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:42.947110    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:44:43.213413 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:43.213327    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:44:56.283034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.283002    2568 generic.go:356] "Generic (PLEG): container finished" podID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerID="46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91" exitCode=0
Sep 19 12:44:56.283461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.283054    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerDied","Data":"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"}
Sep 19 12:44:56.283461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.283453    2568 scope.go:117] "RemoveContainer" containerID="46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"
Sep 19 12:44:56.284607 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.284580    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:56.284670 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.284649    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:56.284706 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:56.284679    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:44:57.291481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:57.291444    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerStarted","Data":"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"}
Sep 19 12:44:57.803290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:57.803246    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:44:57.804641 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:57.804619    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 19 12:44:57.804702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:57.804670    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 19 12:44:58.803729 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:58.803693    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 19 12:44:58.804140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:58.803752    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 19 12:44:59.803272 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:59.803240    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 19 12:44:59.803549 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:44:59.803291    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 19 12:45:00.138907 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.138871    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"]
Sep 19 12:45:00.139290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.139270    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.139290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.139281    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.139368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.139292    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.139368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.139298    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.139368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.139353    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.149130 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.149024    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"]
Sep 19 12:45:00.149328 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.149229    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"
Sep 19 12:45:00.150432 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.150403    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6399c1a-1755-46ea-be8d-ae742ea69995" containerName="istio-proxy"
Sep 19 12:45:00.152802 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.152777    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:45:00.154009 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.153982    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"]
Sep 19 12:45:00.155270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.155248    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"
Sep 19 12:45:00.170417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.170384    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"
Sep 19 12:45:00.174528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.172004    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"]
Sep 19 12:45:00.174528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.172617    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"
Sep 19 12:45:00.186138 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.186111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"]
Sep 19 12:45:00.186389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.186371    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"]
Sep 19 12:45:00.186562 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.186548    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"]
Sep 19 12:45:00.186690 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.186678    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"]
Sep 19 12:45:00.187103 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.187089    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"]
Sep 19 12:45:00.189354 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.189335    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"
Sep 19 12:45:00.190159 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.190142    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"
Sep 19 12:45:00.198421 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.198401    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"
Sep 19 12:45:00.210640 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.210618    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"]
Sep 19 12:45:00.216184 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.216158    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"
Sep 19 12:45:00.242610 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.242574    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"
Sep 19 12:45:00.312757 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.312703    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"
Sep 19 12:45:00.585908 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.583736    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"]
Sep 19 12:45:00.593106 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:00.593076    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d197dd6_15f5_4d17_bd01_7f1884cadf34.slice/crio-eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed WatchSource:0}: Error finding container eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed: Status 404 returned error can't find the container with id eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed
Sep 19 12:45:00.594031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.593894    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"]
Sep 19 12:45:00.594603 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:00.594563    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e9260f_d2fd_4e9b_8a73_eccfcb16d57e.slice/crio-648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53 WatchSource:0}: Error finding container 648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53: Status 404 returned error can't find the container with id 648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53
Sep 19 12:45:00.595477 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:00.595456    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911cff9b_7621_4354_99ee_71e898cf49df.slice/crio-b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236 WatchSource:0}: Error finding container b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236: Status 404 returned error can't find the container with id b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236
Sep 19 12:45:00.596345 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:00.596293    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53069d6c_22da_4289_b9b4_39f6e2568d7c.slice/crio-e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90 WatchSource:0}: Error finding container e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90: Status 404 returned error can't find the container with id e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90
Sep 19 12:45:00.597030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.597014    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"]
Sep 19 12:45:00.599295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.599277    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"]
Sep 19 12:45:00.681520 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.678660    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"]
Sep 19 12:45:00.803534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.803471    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 19 12:45:00.803659 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:00.803557    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 19 12:45:01.123623 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.123593    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:45:01.321113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.321015    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp" event={"ID":"a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e","Type":"ContainerStarted","Data":"bce581ab8b01e9ca33f7a8a4dbe53cbda72534f70d15a2edd38520811035caa3"}
Sep 19 12:45:01.321113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.321084    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp" event={"ID":"a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e","Type":"ContainerStarted","Data":"648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53"}
Sep 19 12:45:01.322699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.322671    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn" event={"ID":"53069d6c-22da-4289-b9b4-39f6e2568d7c","Type":"ContainerStarted","Data":"979bd092bc0a94e75703d8384fd2bb8562f1b2dcfb2dc2d0456a5fa29219f036"}
Sep 19 12:45:01.322749 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.322725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn" event={"ID":"53069d6c-22da-4289-b9b4-39f6e2568d7c","Type":"ContainerStarted","Data":"e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90"}
Sep 19 12:45:01.324302 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.324277    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw" event={"ID":"911cff9b-7621-4354-99ee-71e898cf49df","Type":"ContainerStarted","Data":"edfe8d30b1d33a24d2f51dac346af6e94f6ccc5a69b9a629626be272d9a02fd0"}
Sep 19 12:45:01.324354 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.324329    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw" event={"ID":"911cff9b-7621-4354-99ee-71e898cf49df","Type":"ContainerStarted","Data":"b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236"}
Sep 19 12:45:01.326072 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.326048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f" event={"ID":"9d197dd6-15f5-4d17-bd01-7f1884cadf34","Type":"ContainerStarted","Data":"effe0b34fae6dc8e4ec9ab9b8f72df860de7787d181f81c218abf3868a491a7f"}
Sep 19 12:45:01.326134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.326095    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f" event={"ID":"9d197dd6-15f5-4d17-bd01-7f1884cadf34","Type":"ContainerStarted","Data":"eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed"}
Sep 19 12:45:01.327782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.327761    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67" event={"ID":"b4977a15-00f4-42f0-bad9-fdd8a8a87b4d","Type":"ContainerStarted","Data":"9168b3ab547257dd84757d3859293487ad6e5daf39dd1104244cc91ff29c97ed"}
Sep 19 12:45:01.327850 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.327799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67" event={"ID":"b4977a15-00f4-42f0-bad9-fdd8a8a87b4d","Type":"ContainerStarted","Data":"1c6c9d517fc99460210272d3d2a206de68a9162f0823cc9eb43144dc04d2e6b8"}
Sep 19 12:45:01.360657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.360597    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67" podStartSLOduration=1.360572779 podStartE2EDuration="1.360572779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:01.357730785 +0000 UTC m=+1185.647286555" watchObservedRunningTime="2026-09-19 12:45:01.360572779 +0000 UTC m=+1185.650128549"
Sep 19 12:45:01.361933 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.361853    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp" podStartSLOduration=1.361838003 podStartE2EDuration="1.361838003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:01.343513651 +0000 UTC m=+1185.633069414" watchObservedRunningTime="2026-09-19 12:45:01.361838003 +0000 UTC m=+1185.651393772"
Sep 19 12:45:01.390638 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.390576    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw" podStartSLOduration=1.390553483 podStartE2EDuration="1.390553483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:01.375343056 +0000 UTC m=+1185.664898828" watchObservedRunningTime="2026-09-19 12:45:01.390553483 +0000 UTC m=+1185.680109253"
Sep 19 12:45:01.392110 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.392066    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f" podStartSLOduration=1.392053397 podStartE2EDuration="1.392053397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:01.390858582 +0000 UTC m=+1185.680414353" watchObservedRunningTime="2026-09-19 12:45:01.392053397 +0000 UTC m=+1185.681609165"
Sep 19 12:45:01.406055 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.406011    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn" podStartSLOduration=1.4059904859999999 podStartE2EDuration="1.405990486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:01.405818663 +0000 UTC m=+1185.695374432" watchObservedRunningTime="2026-09-19 12:45:01.405990486 +0000 UTC m=+1185.695546256"
Sep 19 12:45:01.803518 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.803459    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 19 12:45:01.803718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:01.803546    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 19 12:45:02.335487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.335455    2568 generic.go:356] "Generic (PLEG): container finished" podID="a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e" containerID="bce581ab8b01e9ca33f7a8a4dbe53cbda72534f70d15a2edd38520811035caa3" exitCode=0
Sep 19 12:45:02.335957 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.335552    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp" event={"ID":"a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e","Type":"ContainerDied","Data":"bce581ab8b01e9ca33f7a8a4dbe53cbda72534f70d15a2edd38520811035caa3"}
Sep 19 12:45:02.337264 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.337238    2568 generic.go:356] "Generic (PLEG): container finished" podID="53069d6c-22da-4289-b9b4-39f6e2568d7c" containerID="979bd092bc0a94e75703d8384fd2bb8562f1b2dcfb2dc2d0456a5fa29219f036" exitCode=0
Sep 19 12:45:02.337394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.337304    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn" event={"ID":"53069d6c-22da-4289-b9b4-39f6e2568d7c","Type":"ContainerDied","Data":"979bd092bc0a94e75703d8384fd2bb8562f1b2dcfb2dc2d0456a5fa29219f036"}
Sep 19 12:45:02.338889 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.338869    2568 generic.go:356] "Generic (PLEG): container finished" podID="911cff9b-7621-4354-99ee-71e898cf49df" containerID="edfe8d30b1d33a24d2f51dac346af6e94f6ccc5a69b9a629626be272d9a02fd0" exitCode=0
Sep 19 12:45:02.338974 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.338899    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw" event={"ID":"911cff9b-7621-4354-99ee-71e898cf49df","Type":"ContainerDied","Data":"edfe8d30b1d33a24d2f51dac346af6e94f6ccc5a69b9a629626be272d9a02fd0"}
Sep 19 12:45:02.340404 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.340384    2568 generic.go:356] "Generic (PLEG): container finished" podID="9d197dd6-15f5-4d17-bd01-7f1884cadf34" containerID="effe0b34fae6dc8e4ec9ab9b8f72df860de7787d181f81c218abf3868a491a7f" exitCode=0
Sep 19 12:45:02.340479 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.340448    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f" event={"ID":"9d197dd6-15f5-4d17-bd01-7f1884cadf34","Type":"ContainerDied","Data":"effe0b34fae6dc8e4ec9ab9b8f72df860de7787d181f81c218abf3868a491a7f"}
Sep 19 12:45:02.341873 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.341853    2568 generic.go:356] "Generic (PLEG): container finished" podID="b4977a15-00f4-42f0-bad9-fdd8a8a87b4d" containerID="9168b3ab547257dd84757d3859293487ad6e5daf39dd1104244cc91ff29c97ed" exitCode=0
Sep 19 12:45:02.341939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.341922    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67" event={"ID":"b4977a15-00f4-42f0-bad9-fdd8a8a87b4d","Type":"ContainerDied","Data":"9168b3ab547257dd84757d3859293487ad6e5daf39dd1104244cc91ff29c97ed"}
Sep 19 12:45:02.807567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.807532    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:45:02.808853 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:02.808826    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:45:03.502807 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:03.502782    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"
Sep 19 12:45:03.557188 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:03.557155    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"
Sep 19 12:45:03.612581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:03.612555    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"
Sep 19 12:45:03.615846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:03.615824    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"
Sep 19 12:45:03.619014 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:03.618995    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"
Sep 19 12:45:04.357549 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.357520    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"
Sep 19 12:45:04.357743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.357520    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn" event={"ID":"53069d6c-22da-4289-b9b4-39f6e2568d7c","Type":"ContainerDied","Data":"e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90"}
Sep 19 12:45:04.357743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.357670    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e58e6d201f712efadb444a7792605d32cd6f1ba052c6c24fd6973d47f412dd90"
Sep 19 12:45:04.358837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.358788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw" event={"ID":"911cff9b-7621-4354-99ee-71e898cf49df","Type":"ContainerDied","Data":"b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236"}
Sep 19 12:45:04.358837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.358834    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0fc8d6c5572763964b502807c80a98b5eb007093e43007cfa54dedb07d67236"
Sep 19 12:45:04.359053 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.358798    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830365-ndpqw"
Sep 19 12:45:04.360215 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.360197    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"
Sep 19 12:45:04.360215 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.360203    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f" event={"ID":"9d197dd6-15f5-4d17-bd01-7f1884cadf34","Type":"ContainerDied","Data":"eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed"}
Sep 19 12:45:04.360381 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.360234    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb1dbfd7364eedccfe288f6b5bb58e764a2c39b7e4302d62eca7d9b0020feaed"
Sep 19 12:45:04.361626 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.361607    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"
Sep 19 12:45:04.361626 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.361614    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67" event={"ID":"b4977a15-00f4-42f0-bad9-fdd8a8a87b4d","Type":"ContainerDied","Data":"1c6c9d517fc99460210272d3d2a206de68a9162f0823cc9eb43144dc04d2e6b8"}
Sep 19 12:45:04.361784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.361652    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c6c9d517fc99460210272d3d2a206de68a9162f0823cc9eb43144dc04d2e6b8"
Sep 19 12:45:04.363065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.363039    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"
Sep 19 12:45:04.363166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.363041    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp" event={"ID":"a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e","Type":"ContainerDied","Data":"648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53"}
Sep 19 12:45:04.363229 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:04.363167    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="648fad39d83d78e36f24a701e4ca57dc55f48f34f7418cfabc8e6b483242be53"
Sep 19 12:45:11.694364 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694333    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:45:11.694941 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694921    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d197dd6-15f5-4d17-bd01-7f1884cadf34" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694943    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d197dd6-15f5-4d17-bd01-7f1884cadf34" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694961    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="911cff9b-7621-4354-99ee-71e898cf49df" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694969    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="911cff9b-7621-4354-99ee-71e898cf49df" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694984    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4977a15-00f4-42f0-bad9-fdd8a8a87b4d" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.694992    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4977a15-00f4-42f0-bad9-fdd8a8a87b4d" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695009    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e" containerName="cleanup"
Sep 19 12:45:11.695031 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695017    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695041    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53069d6c-22da-4289-b9b4-39f6e2568d7c" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695049    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="53069d6c-22da-4289-b9b4-39f6e2568d7c" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695154    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695166    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d197dd6-15f5-4d17-bd01-7f1884cadf34" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695177    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="53069d6c-22da-4289-b9b4-39f6e2568d7c" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695188    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4977a15-00f4-42f0-bad9-fdd8a8a87b4d" containerName="cleanup"
Sep 19 12:45:11.695375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.695196    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="911cff9b-7621-4354-99ee-71e898cf49df" containerName="cleanup"
Sep 19 12:45:11.698553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.698480    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.700975 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.700955    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9a75910-openshift-default-dockercfg-6cfq7\""
Sep 19 12:45:11.713875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.713849    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:45:11.778704 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.778875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.778875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.778875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.778875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778839    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.778875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.778859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.779110 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.779028    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.779172 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.779110    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.779172 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.779148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cds87\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880143    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880237    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880270    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880416    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880450    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cds87\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880720    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880794 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880764    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.880932 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.880910    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.881045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.881026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.881111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.881093    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.882752 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.882730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.882846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.882764    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.889610 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.889582    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cds87\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:11.889703 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:11.889625    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token\") pod \"e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") " pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:12.009210 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.009128    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:12.285113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.285088    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:45:12.286874 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:12.286845    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3814fca7_0020_47c1_992f_bb74da538c10.slice/crio-3fdf1b8e17c00cacbfabeab42f84ef311b21e3443e6ddcd64e51890916700d20 WatchSource:0}: Error finding container 3fdf1b8e17c00cacbfabeab42f84ef311b21e3443e6ddcd64e51890916700d20: Status 404 returned error can't find the container with id 3fdf1b8e17c00cacbfabeab42f84ef311b21e3443e6ddcd64e51890916700d20
Sep 19 12:45:12.289552 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.289511    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:12.289628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.289611    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:12.289674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.289659    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:12.418989 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.418946    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerStarted","Data":"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"}
Sep 19 12:45:12.419143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.419011    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerStarted","Data":"3fdf1b8e17c00cacbfabeab42f84ef311b21e3443e6ddcd64e51890916700d20"}
Sep 19 12:45:12.438007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:12.437954    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podStartSLOduration=1.437931197 podStartE2EDuration="1.437931197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:12.436859037 +0000 UTC m=+1196.726414807" watchObservedRunningTime="2026-09-19 12:45:12.437931197 +0000 UTC m=+1196.727486965"
Sep 19 12:45:13.009570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:13.009533    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:13.015347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:13.015321    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:13.424254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:13.424223    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:13.425266 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:13.425251    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:17.707975 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:17.707948    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:45:17.708446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:17.708274    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:45:17.785701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:17.785670    2568 scope.go:117] "RemoveContainer" containerID="07c312e8e7503e06ad15ded1e51392921d4239eb3d28cb4569d5b2d777067aa6"
Sep 19 12:45:20.609232 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.609182    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:45:20.614634 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.614611    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.617005 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.616986    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e9a75910-dockercfg-6q2gv\""
Sep 19 12:45:20.617891 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.617876    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e9a75910\""
Sep 19 12:45:20.626997 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.626975    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:45:20.640120 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.640091    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:45:20.643786 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.643769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.655064 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.655036    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:45:20.671964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.671932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.672144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.671981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zsqh\" (UniqueName: \"kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.672144 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.672079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.672266 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.672192    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cmkp\" (UniqueName: \"kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.773074 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773034    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.773274 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zsqh\" (UniqueName: \"kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.773274 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773138    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.773274 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773217    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cmkp\" (UniqueName: \"kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.773868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773848    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.773927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.773875    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.781678 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.781649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zsqh\" (UniqueName: \"kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh\") pod \"payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.781913 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.781893    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cmkp\" (UniqueName: \"kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp\") pod \"payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") " pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:20.925395 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.925353    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:20.957101 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:20.957064    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:21.201718 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.199162    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:45:21.239754 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.239727    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:45:21.241562 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:45:21.241540    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c1af05_f903_4a59_a94c_35c2f4179a78.slice/crio-ffad3e6b748fab4214a09e6b2deda01bbe0957880e6fb7289868c2c712126c8b WatchSource:0}: Error finding container ffad3e6b748fab4214a09e6b2deda01bbe0957880e6fb7289868c2c712126c8b: Status 404 returned error can't find the container with id ffad3e6b748fab4214a09e6b2deda01bbe0957880e6fb7289868c2c712126c8b
Sep 19 12:45:21.472838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.472731    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" event={"ID":"28c1af05-f903-4a59-a94c-35c2f4179a78","Type":"ContainerStarted","Data":"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"}
Sep 19 12:45:21.472838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.472800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" event={"ID":"28c1af05-f903-4a59-a94c-35c2f4179a78","Type":"ContainerStarted","Data":"ffad3e6b748fab4214a09e6b2deda01bbe0957880e6fb7289868c2c712126c8b"}
Sep 19 12:45:21.473067 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.472969    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:21.474551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.474521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" event={"ID":"dade702b-d547-4fe3-9ee2-29e1c557c278","Type":"ContainerStarted","Data":"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"}
Sep 19 12:45:21.474681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.474619    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:21.474681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.474646    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" event={"ID":"dade702b-d547-4fe3-9ee2-29e1c557c278","Type":"ContainerStarted","Data":"0a0becc96616a3a97504828cb493d2747ae162d907745f49f7c7ee89df13d5bc"}
Sep 19 12:45:21.493885 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.493828    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" podStartSLOduration=1.4938091199999999 podStartE2EDuration="1.49380912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:21.490000615 +0000 UTC m=+1205.779556385" watchObservedRunningTime="2026-09-19 12:45:21.49380912 +0000 UTC m=+1205.783364896"
Sep 19 12:45:21.514904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:21.514856    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" podStartSLOduration=1.514839485 podStartE2EDuration="1.514839485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:21.513617565 +0000 UTC m=+1205.803173339" watchObservedRunningTime="2026-09-19 12:45:21.514839485 +0000 UTC m=+1205.804395251"
Sep 19 12:45:32.483755 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:32.483719    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:45:32.485666 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:32.485638    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:45:40.600117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.600081    2568 generic.go:356] "Generic (PLEG): container finished" podID="3814fca7-0020-47c1-992f-bb74da538c10" containerID="8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f" exitCode=0
Sep 19 12:45:40.600577 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.600150    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerDied","Data":"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"}
Sep 19 12:45:40.600577 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.600573    2568 scope.go:117] "RemoveContainer" containerID="8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"
Sep 19 12:45:40.601836 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.601807    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:40.601904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.601876    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:40.601942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:40.601927    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:45:41.607775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:41.607739    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerStarted","Data":"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"}
Sep 19 12:45:42.009414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:42.009382    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:42.010719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:42.010695    2568 patch_prober.go:28] interesting pod/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 19 12:45:42.010840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:42.010749    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 19 12:45:43.009452 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:43.009419    2568 patch_prober.go:28] interesting pod/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 19 12:45:43.009871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:43.009469    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 19 12:45:43.424244 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:43.424188    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:44.010062 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:44.010017    2568 patch_prober.go:28] interesting pod/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 19 12:45:44.010574 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:44.010090    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 19 12:45:45.009757 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:45.009721    2568 patch_prober.go:28] interesting pod/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 19 12:45:45.009948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:45.009777    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 19 12:45:46.010284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:46.010249    2568 patch_prober.go:28] interesting pod/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 19 12:45:46.010703 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:46.010301    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 19 12:45:47.018347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:47.018314    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:45:47.019605 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:45:47.019587    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:46:08.095914 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.095834    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"]
Sep 19 12:46:08.099291 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.099275    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:08.101594 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.101547    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:46:08.101705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.101676    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:46:08.105852 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.105827    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"]
Sep 19 12:46:08.273746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.273709    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-61583b-94j2q\" (UID: \"3c87ec0a-b8bd-403f-940b-a1d27aa3b423\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:08.375187 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.375087    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-61583b-94j2q\" (UID: \"3c87ec0a-b8bd-403f-940b-a1d27aa3b423\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:08.375756 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.375737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-61583b-94j2q\" (UID: \"3c87ec0a-b8bd-403f-940b-a1d27aa3b423\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:08.411125 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.411097    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:08.685341 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.685315    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"]
Sep 19 12:46:08.687346 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:46:08.687321    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c87ec0a_b8bd_403f_940b_a1d27aa3b423.slice/crio-dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2 WatchSource:0}: Error finding container dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2: Status 404 returned error can't find the container with id dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2
Sep 19 12:46:08.768543 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:08.768486    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q" event={"ID":"3c87ec0a-b8bd-403f-940b-a1d27aa3b423","Type":"ContainerStarted","Data":"dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2"}
Sep 19 12:46:09.775442 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:09.775406    2568 generic.go:356] "Generic (PLEG): container finished" podID="3c87ec0a-b8bd-403f-940b-a1d27aa3b423" containerID="8506b1e0dbf1ffc534ed5434039bd5a2b82c286eba511e166fa5fe52e2ba8fe7" exitCode=0
Sep 19 12:46:09.775870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:09.775488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q" event={"ID":"3c87ec0a-b8bd-403f-940b-a1d27aa3b423","Type":"ContainerDied","Data":"8506b1e0dbf1ffc534ed5434039bd5a2b82c286eba511e166fa5fe52e2ba8fe7"}
Sep 19 12:46:10.931488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:10.931461    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:11.103833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.103739    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca\") pod \"3c87ec0a-b8bd-403f-940b-a1d27aa3b423\" (UID: \"3c87ec0a-b8bd-403f-940b-a1d27aa3b423\") "
Sep 19 12:46:11.104141 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.104117    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c87ec0a-b8bd-403f-940b-a1d27aa3b423" (UID: "3c87ec0a-b8bd-403f-940b-a1d27aa3b423"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:11.205115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.205084    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c87ec0a-b8bd-403f-940b-a1d27aa3b423-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:11.787989 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.787957    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"
Sep 19 12:46:11.788162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.787996    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q" event={"ID":"3c87ec0a-b8bd-403f-940b-a1d27aa3b423","Type":"ContainerDied","Data":"dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2"}
Sep 19 12:46:11.788162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:11.788040    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc47748b85544e250debc2284912f7595c7b1c3f71197b5a4348c78b9a1f55a2"
Sep 19 12:46:17.810717 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:17.810682    2568 scope.go:117] "RemoveContainer" containerID="ab67aa1a31fae827c2f34d60efecae85d4953d7fc898f6f1d6ec48ea3ad68806"
Sep 19 12:46:17.820517 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:17.820479    2568 scope.go:117] "RemoveContainer" containerID="7b6a5efecd07e8da5b4eae7c8aa1c2465281a07ce02d8b64b93b0b9f4f3c6943"
Sep 19 12:46:17.828436 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:17.828415    2568 scope.go:117] "RemoveContainer" containerID="e87ec8c98a606838296dec8e67b9b7c1bc83a0577a1f61e72e9d537d613b6eaa"
Sep 19 12:46:18.662328 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.662293    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"]
Sep 19 12:46:18.664895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.664868    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-61583b-29830365-6jf67"]
Sep 19 12:46:18.700694 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.700664    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:46:18.700955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.700914    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" podUID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" containerName="payload-processing" containerID="cri-o://80daf47148d6f23ab6567befc0dc57ff61138ab2f1bc0ba028d3063de56673d4" gracePeriod=30
Sep 19 12:46:18.707267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.707178    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:46:18.707447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.707394    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" podUID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" containerName="payload-pre-processing" containerID="cri-o://2cf273d622804e953ebac17fcc7d6c833363ddee6fa45475cc6ac65d790a6370" gracePeriod=30
Sep 19 12:46:18.829837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.829795    2568 generic.go:356] "Generic (PLEG): container finished" podID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" containerID="2cf273d622804e953ebac17fcc7d6c833363ddee6fa45475cc6ac65d790a6370" exitCode=0
Sep 19 12:46:18.830248 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.829865    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" event={"ID":"f90e03e1-efe7-4d6a-9436-feb4c4840fd7","Type":"ContainerDied","Data":"2cf273d622804e953ebac17fcc7d6c833363ddee6fa45475cc6ac65d790a6370"}
Sep 19 12:46:18.832004 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.831984    2568 generic.go:356] "Generic (PLEG): container finished" podID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" containerID="80daf47148d6f23ab6567befc0dc57ff61138ab2f1bc0ba028d3063de56673d4" exitCode=0
Sep 19 12:46:18.832117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.832054    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" event={"ID":"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd","Type":"ContainerDied","Data":"80daf47148d6f23ab6567befc0dc57ff61138ab2f1bc0ba028d3063de56673d4"}
Sep 19 12:46:18.885426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.885384    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"]
Sep 19 12:46:18.888017 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:18.887983    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-61583b-94j2q"]
Sep 19 12:46:19.033783 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.033754    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:46:19.036814 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.036793    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:46:19.084764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.084725    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume\") pod \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") "
Sep 19 12:46:19.084951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.084812    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j885p\" (UniqueName: \"kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p\") pod \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\" (UID: \"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd\") "
Sep 19 12:46:19.084951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.084874    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume\") pod \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") "
Sep 19 12:46:19.084951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.084904    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtx9k\" (UniqueName: \"kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k\") pod \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\" (UID: \"f90e03e1-efe7-4d6a-9436-feb4c4840fd7\") "
Sep 19 12:46:19.085137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.085096    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" (UID: "88da8cce-4e41-4a3f-ab34-c4679e6d2bbd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:19.085306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.085282    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f90e03e1-efe7-4d6a-9436-feb4c4840fd7" (UID: "f90e03e1-efe7-4d6a-9436-feb4c4840fd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:19.085386 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.085307    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:19.087008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.086980    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p" (OuterVolumeSpecName: "kube-api-access-j885p") pod "88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" (UID: "88da8cce-4e41-4a3f-ab34-c4679e6d2bbd"). InnerVolumeSpecName "kube-api-access-j885p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:19.087008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.086979    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k" (OuterVolumeSpecName: "kube-api-access-mtx9k") pod "f90e03e1-efe7-4d6a-9436-feb4c4840fd7" (UID: "f90e03e1-efe7-4d6a-9436-feb4c4840fd7"). InnerVolumeSpecName "kube-api-access-mtx9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:19.186840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.186790    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:19.186840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.186834    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtx9k\" (UniqueName: \"kubernetes.io/projected/f90e03e1-efe7-4d6a-9436-feb4c4840fd7-kube-api-access-mtx9k\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:19.186840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.186846    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j885p\" (UniqueName: \"kubernetes.io/projected/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd-kube-api-access-j885p\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:19.724429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.724396    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c87ec0a-b8bd-403f-940b-a1d27aa3b423" path="/var/lib/kubelet/pods/3c87ec0a-b8bd-403f-940b-a1d27aa3b423/volumes"
Sep 19 12:46:19.724810 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.724794    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4977a15-00f4-42f0-bad9-fdd8a8a87b4d" path="/var/lib/kubelet/pods/b4977a15-00f4-42f0-bad9-fdd8a8a87b4d/volumes"
Sep 19 12:46:19.838848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.838816    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"
Sep 19 12:46:19.839319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.838850    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs" event={"ID":"88da8cce-4e41-4a3f-ab34-c4679e6d2bbd","Type":"ContainerDied","Data":"47af366a6c0e0657dec11cae82038b7d7d9b836c8e7a4e9cab7551e070004113"}
Sep 19 12:46:19.839319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.838913    2568 scope.go:117] "RemoveContainer" containerID="80daf47148d6f23ab6567befc0dc57ff61138ab2f1bc0ba028d3063de56673d4"
Sep 19 12:46:19.840536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.840515    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs" event={"ID":"f90e03e1-efe7-4d6a-9436-feb4c4840fd7","Type":"ContainerDied","Data":"da94a017abd28d48de747eb7e7d7bb87e61635c2016c1388ae1c7a00e14593d5"}
Sep 19 12:46:19.840536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.840533    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"
Sep 19 12:46:19.848061 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.848045    2568 scope.go:117] "RemoveContainer" containerID="2cf273d622804e953ebac17fcc7d6c833363ddee6fa45475cc6ac65d790a6370"
Sep 19 12:46:19.860656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.860625    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:46:19.863077 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.863054    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-61583b-65cc6c6cc9-2wxqs"]
Sep 19 12:46:19.875803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.875781    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:46:19.880183 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:19.880164    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-61583b-7d4686d566-knxzs"]
Sep 19 12:46:21.722100 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:21.722064    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" path="/var/lib/kubelet/pods/88da8cce-4e41-4a3f-ab34-c4679e6d2bbd/volumes"
Sep 19 12:46:21.722486 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:21.722431    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" path="/var/lib/kubelet/pods/f90e03e1-efe7-4d6a-9436-feb4c4840fd7/volumes"
Sep 19 12:46:27.435097 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.435058    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:46:27.435484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.435259    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" containerID="cri-o://7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b" gracePeriod=30
Sep 19 12:46:27.774199 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774101    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"]
Sep 19 12:46:27.774609 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774591    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" containerName="payload-pre-processing"
Sep 19 12:46:27.774705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774611    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" containerName="payload-pre-processing"
Sep 19 12:46:27.774705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774627    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c87ec0a-b8bd-403f-940b-a1d27aa3b423" containerName="revoke-keys"
Sep 19 12:46:27.774705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774635    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c87ec0a-b8bd-403f-940b-a1d27aa3b423" containerName="revoke-keys"
Sep 19 12:46:27.774705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774649    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" containerName="payload-processing"
Sep 19 12:46:27.774705 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774657    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" containerName="payload-processing"
Sep 19 12:46:27.774964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774796    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="88da8cce-4e41-4a3f-ab34-c4679e6d2bbd" containerName="payload-processing"
Sep 19 12:46:27.774964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774809    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c87ec0a-b8bd-403f-940b-a1d27aa3b423" containerName="revoke-keys"
Sep 19 12:46:27.774964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.774824    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f90e03e1-efe7-4d6a-9436-feb4c4840fd7" containerName="payload-pre-processing"
Sep 19 12:46:27.779366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.779346    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:27.784146 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.784120    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:46:27.784485 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.784465    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:46:27.787325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.787299    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"]
Sep 19 12:46:27.869374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.869335    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf\" (UID: \"4e17c340-8041-4efe-8e2a-40b637cfe715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:27.970406 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.970374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf\" (UID: \"4e17c340-8041-4efe-8e2a-40b637cfe715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:27.970973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:27.970954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf\" (UID: \"4e17c340-8041-4efe-8e2a-40b637cfe715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:28.090875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:28.090793    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:28.364455 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:28.364434    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"]
Sep 19 12:46:28.366430 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:46:28.366399    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e17c340_8041_4efe_8e2a_40b637cfe715.slice/crio-3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5 WatchSource:0}: Error finding container 3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5: Status 404 returned error can't find the container with id 3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5
Sep 19 12:46:28.896387 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:28.896355    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf" event={"ID":"4e17c340-8041-4efe-8e2a-40b637cfe715","Type":"ContainerStarted","Data":"c7896147e768116417675d92e3631a5a525d995e4160f23bf42b1701524a7737"}
Sep 19 12:46:28.896804 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:28.896417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf" event={"ID":"4e17c340-8041-4efe-8e2a-40b637cfe715","Type":"ContainerStarted","Data":"3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5"}
Sep 19 12:46:28.914269 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:28.914216    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf" podStartSLOduration=1.914200867 podStartE2EDuration="1.914200867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:28.912313335 +0000 UTC m=+1273.201869104" watchObservedRunningTime="2026-09-19 12:46:28.914200867 +0000 UTC m=+1273.203756682"
Sep 19 12:46:29.903136 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:29.903101    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e17c340-8041-4efe-8e2a-40b637cfe715" containerID="c7896147e768116417675d92e3631a5a525d995e4160f23bf42b1701524a7737" exitCode=0
Sep 19 12:46:29.903558 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:29.903192    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf" event={"ID":"4e17c340-8041-4efe-8e2a-40b637cfe715","Type":"ContainerDied","Data":"c7896147e768116417675d92e3631a5a525d995e4160f23bf42b1701524a7737"}
Sep 19 12:46:30.469367 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:30.469330    2568 patch_prober.go:28] interesting pod/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 12:46:30.469602 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:30.469412    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:46:31.016553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.016520    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:46:31.017014 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.016758    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" podUID="fe172484-05d1-4e64-8f82-10e10a3faa05" containerName="istio-proxy" containerID="cri-o://76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4" gracePeriod=30
Sep 19 12:46:31.057865 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.057842    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:31.101559 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.101527    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca\") pod \"4e17c340-8041-4efe-8e2a-40b637cfe715\" (UID: \"4e17c340-8041-4efe-8e2a-40b637cfe715\") "
Sep 19 12:46:31.101932 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.101905    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e17c340-8041-4efe-8e2a-40b637cfe715" (UID: "4e17c340-8041-4efe-8e2a-40b637cfe715"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:31.202852 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.202768    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e17c340-8041-4efe-8e2a-40b637cfe715-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:31.916665 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.916635    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"
Sep 19 12:46:31.916838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.916672    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf" event={"ID":"4e17c340-8041-4efe-8e2a-40b637cfe715","Type":"ContainerDied","Data":"3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5"}
Sep 19 12:46:31.916838 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:31.916718    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c3fa5bf4f82a997c73688200de5f46682a6df21a07bcfe868588de6df29f6b5"
Sep 19 12:46:32.692665 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.692637    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.818735    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.818809    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.818854    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.818884    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.818940    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819003    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819040    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819085    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819104    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwh7t\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.819534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819183    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket\") pod \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\" (UID: \"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6\") "
Sep 19 12:46:32.820147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819722    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.820147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.819986    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:32.821897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.821646    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:32.823112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.822928    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:32.823112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.823069    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:32.823747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.823706    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:32.826611 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.825722    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t" (OuterVolumeSpecName: "kube-api-access-wwh7t") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "kube-api-access-wwh7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:32.826611 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.826022    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:46:32.826780 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.826608    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" (UID: "e0e30b02-fbee-40fb-aa51-ba48f1e1acf6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:32.920491 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920460    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwh7t\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-kube-api-access-wwh7t\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920491 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920487    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920491 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920521    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920531    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920539    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920546    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920555    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.920763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.920563    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:32.924774 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.924746    2568 generic.go:356] "Generic (PLEG): container finished" podID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerID="7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b" exitCode=0
Sep 19 12:46:32.924929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.924801    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"
Sep 19 12:46:32.924929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.924822    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerDied","Data":"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"}
Sep 19 12:46:32.924929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.924878    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv" event={"ID":"e0e30b02-fbee-40fb-aa51-ba48f1e1acf6","Type":"ContainerDied","Data":"3473b2650069a3b6969c93df7f5764d8b48a97f38e33ac7d08c43638347e5a1f"}
Sep 19 12:46:32.924929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.924902    2568 scope.go:117] "RemoveContainer" containerID="7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"
Sep 19 12:46:32.934767 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.934751    2568 scope.go:117] "RemoveContainer" containerID="fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"
Sep 19 12:46:32.943142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.943124    2568 scope.go:117] "RemoveContainer" containerID="7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"
Sep 19 12:46:32.943379 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:46:32.943361    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b\": container with ID starting with 7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b not found: ID does not exist" containerID="7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"
Sep 19 12:46:32.943428 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.943387    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b"} err="failed to get container status \"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b\": rpc error: code = NotFound desc = could not find container \"7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b\": container with ID starting with 7afaea0a1aa7ccd481cf71e2e1ec2e213b85816bdf0479a39375495ed8f55d0b not found: ID does not exist"
Sep 19 12:46:32.943428 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.943403    2568 scope.go:117] "RemoveContainer" containerID="fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"
Sep 19 12:46:32.943640 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:46:32.943626    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654\": container with ID starting with fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654 not found: ID does not exist" containerID="fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"
Sep 19 12:46:32.943682 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.943644    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654"} err="failed to get container status \"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654\": rpc error: code = NotFound desc = could not find container \"fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654\": container with ID starting with fdf1f9d726a5bc41c0bafee6cd220a5c7c4b4e5eefd15a808e53eed4ba697654 not found: ID does not exist"
Sep 19 12:46:32.948955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.948934    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:46:32.954592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:32.954570    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-61583b-openshift-default-6b8c5fdcdd-c6dlv"]
Sep 19 12:46:33.725177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:33.725144    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" path="/var/lib/kubelet/pods/e0e30b02-fbee-40fb-aa51-ba48f1e1acf6/volumes"
Sep 19 12:46:36.270147 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.270124    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:46:36.355544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355435    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355486    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355542    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jm9p\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355579    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355623    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355653    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355698    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355730    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.355784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355782    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert\") pod \"fe172484-05d1-4e64-8f82-10e10a3faa05\" (UID: \"fe172484-05d1-4e64-8f82-10e10a3faa05\") "
Sep 19 12:46:36.356088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355788    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:36.356088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355929    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:36.356088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.355939    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:36.356313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.356170    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:36.356313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.356188    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.356313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.356210    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.356313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.356226    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.356313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.356235    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:36.357853 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.357826    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p" (OuterVolumeSpecName: "kube-api-access-6jm9p") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "kube-api-access-6jm9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:36.357969 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.357892    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:36.358177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.358155    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:46:36.358287 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.358159    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe172484-05d1-4e64-8f82-10e10a3faa05" (UID: "fe172484-05d1-4e64-8f82-10e10a3faa05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:36.457196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457140    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.457196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457188    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.457196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457197    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe172484-05d1-4e64-8f82-10e10a3faa05-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.457196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457205    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.457196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457213    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jm9p\" (UniqueName: \"kubernetes.io/projected/fe172484-05d1-4e64-8f82-10e10a3faa05-kube-api-access-6jm9p\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.457536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.457223    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe172484-05d1-4e64-8f82-10e10a3faa05-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:36.948768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.948735    2568 generic.go:356] "Generic (PLEG): container finished" podID="fe172484-05d1-4e64-8f82-10e10a3faa05" containerID="76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4" exitCode=0
Sep 19 12:46:36.948954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.948799    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"
Sep 19 12:46:36.948954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.948821    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" event={"ID":"fe172484-05d1-4e64-8f82-10e10a3faa05","Type":"ContainerDied","Data":"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"}
Sep 19 12:46:36.948954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.948873    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx" event={"ID":"fe172484-05d1-4e64-8f82-10e10a3faa05","Type":"ContainerDied","Data":"50a2ec1bbd9a7e93ae384f660a2086322c8a25630cfaf52d55bf3099a8e802e3"}
Sep 19 12:46:36.948954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.948894    2568 scope.go:117] "RemoveContainer" containerID="76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"
Sep 19 12:46:36.958347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.958330    2568 scope.go:117] "RemoveContainer" containerID="76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"
Sep 19 12:46:36.958643 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:46:36.958624    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4\": container with ID starting with 76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4 not found: ID does not exist" containerID="76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"
Sep 19 12:46:36.958699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.958651    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4"} err="failed to get container status \"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4\": rpc error: code = NotFound desc = could not find container \"76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4\": container with ID starting with 76a4bcf177882d19e56a6eaafacaa70368a7a0e97c49f16c77014a0b347e55b4 not found: ID does not exist"
Sep 19 12:46:36.981246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.981221    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:46:36.983133 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:36.983114    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-e8f833-openshift-default-574c6d4c6b-zstvx"]
Sep 19 12:46:37.722305 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:37.722269    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe172484-05d1-4e64-8f82-10e10a3faa05" path="/var/lib/kubelet/pods/fe172484-05d1-4e64-8f82-10e10a3faa05/volumes"
Sep 19 12:46:38.290842 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.290805    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"]
Sep 19 12:46:38.292892 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.292867    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5be349-29830365-shq5f"]
Sep 19 12:46:38.314547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.314487    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:46:38.314833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.314769    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" podUID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" containerName="payload-processing" containerID="cri-o://36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7" gracePeriod=30
Sep 19 12:46:38.329256 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.329229    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:46:38.329693 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.329662    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" podUID="9bb2647a-5179-4b44-bda7-59962133486e" containerName="payload-pre-processing" containerID="cri-o://1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e" gracePeriod=30
Sep 19 12:46:38.497414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.497381    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"]
Sep 19 12:46:38.500896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.500871    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5be349-v9ghf"]
Sep 19 12:46:38.607326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.607302    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:46:38.610033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.610014    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:46:38.680340 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.680306    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvj86\" (UniqueName: \"kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86\") pod \"9bb2647a-5179-4b44-bda7-59962133486e\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") "
Sep 19 12:46:38.680581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.680389    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpjf2\" (UniqueName: \"kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2\") pod \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") "
Sep 19 12:46:38.680581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.680440    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume\") pod \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\" (UID: \"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f\") "
Sep 19 12:46:38.680581 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.680561    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume\") pod \"9bb2647a-5179-4b44-bda7-59962133486e\" (UID: \"9bb2647a-5179-4b44-bda7-59962133486e\") "
Sep 19 12:46:38.681241 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.681213    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bb2647a-5179-4b44-bda7-59962133486e" (UID: "9bb2647a-5179-4b44-bda7-59962133486e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:38.681420 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.681257    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" (UID: "a9e3f6ee-b6ec-4899-897a-b81eab56dc3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:38.685598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.685546    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86" (OuterVolumeSpecName: "kube-api-access-qvj86") pod "9bb2647a-5179-4b44-bda7-59962133486e" (UID: "9bb2647a-5179-4b44-bda7-59962133486e"). InnerVolumeSpecName "kube-api-access-qvj86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:38.687418 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.687375    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2" (OuterVolumeSpecName: "kube-api-access-hpjf2") pod "a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" (UID: "a9e3f6ee-b6ec-4899-897a-b81eab56dc3f"). InnerVolumeSpecName "kube-api-access-hpjf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:38.782709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.782465    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpjf2\" (UniqueName: \"kubernetes.io/projected/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-kube-api-access-hpjf2\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.782709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.782539    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.782709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.782555    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bb2647a-5179-4b44-bda7-59962133486e-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.782709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.782570    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvj86\" (UniqueName: \"kubernetes.io/projected/9bb2647a-5179-4b44-bda7-59962133486e-kube-api-access-qvj86\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.962131 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.962091    2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb2647a-5179-4b44-bda7-59962133486e" containerID="1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e" exitCode=0
Sep 19 12:46:38.962295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.962140    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"
Sep 19 12:46:38.962295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.962178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" event={"ID":"9bb2647a-5179-4b44-bda7-59962133486e","Type":"ContainerDied","Data":"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"}
Sep 19 12:46:38.962295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.962248    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv" event={"ID":"9bb2647a-5179-4b44-bda7-59962133486e","Type":"ContainerDied","Data":"e502e165cde4bca5ebecb1f31a43bf30cec0e78866ffd5b65bbe51243f74431c"}
Sep 19 12:46:38.962295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.962287    2568 scope.go:117] "RemoveContainer" containerID="1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"
Sep 19 12:46:38.963578 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.963556    2568 generic.go:356] "Generic (PLEG): container finished" podID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" containerID="36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7" exitCode=0
Sep 19 12:46:38.963680 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.963609    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"
Sep 19 12:46:38.963680 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.963636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" event={"ID":"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f","Type":"ContainerDied","Data":"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"}
Sep 19 12:46:38.963771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.963695    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4" event={"ID":"a9e3f6ee-b6ec-4899-897a-b81eab56dc3f","Type":"ContainerDied","Data":"b324de44ed60f2b3659ea0f96700f18d4842bcc5955ce317bd656bd1ff39a665"}
Sep 19 12:46:38.972659 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.972642    2568 scope.go:117] "RemoveContainer" containerID="1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"
Sep 19 12:46:38.972931 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:46:38.972912    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e\": container with ID starting with 1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e not found: ID does not exist" containerID="1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"
Sep 19 12:46:38.973015 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.972938    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e"} err="failed to get container status \"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e\": rpc error: code = NotFound desc = could not find container \"1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e\": container with ID starting with 1a489cb220647f1a86727790af78b455760d5cd278c49b8a29e9c92781ac8e4e not found: ID does not exist"
Sep 19 12:46:38.973015 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.972956    2568 scope.go:117] "RemoveContainer" containerID="36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"
Sep 19 12:46:38.982078 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.982061    2568 scope.go:117] "RemoveContainer" containerID="36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"
Sep 19 12:46:38.982394 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:46:38.982367    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7\": container with ID starting with 36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7 not found: ID does not exist" containerID="36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"
Sep 19 12:46:38.982478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.982401    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7"} err="failed to get container status \"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7\": rpc error: code = NotFound desc = could not find container \"36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7\": container with ID starting with 36c21287ddb5b442dd436dd521067f99bcf3d1e995110cd6ad44ffc5504a35c7 not found: ID does not exist"
Sep 19 12:46:38.989366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.989347    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:46:38.995534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:38.995511    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5be349-6475d864f8-jt2zv"]
Sep 19 12:46:39.009038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.009012    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:46:39.012224 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.012204    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5be349-77cb7c5f9f-gknq4"]
Sep 19 12:46:39.721825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.721790    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e17c340-8041-4efe-8e2a-40b637cfe715" path="/var/lib/kubelet/pods/4e17c340-8041-4efe-8e2a-40b637cfe715/volumes"
Sep 19 12:46:39.722163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.722149    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb2647a-5179-4b44-bda7-59962133486e" path="/var/lib/kubelet/pods/9bb2647a-5179-4b44-bda7-59962133486e/volumes"
Sep 19 12:46:39.722406 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.722395    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d197dd6-15f5-4d17-bd01-7f1884cadf34" path="/var/lib/kubelet/pods/9d197dd6-15f5-4d17-bd01-7f1884cadf34/volumes"
Sep 19 12:46:39.722724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:39.722712    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" path="/var/lib/kubelet/pods/a9e3f6ee-b6ec-4899-897a-b81eab56dc3f/volumes"
Sep 19 12:46:46.209396 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.209361    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 12:46:46.209962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.209937    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17c340-8041-4efe-8e2a-40b637cfe715" containerName="revoke-keys"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.209970    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17c340-8041-4efe-8e2a-40b637cfe715" containerName="revoke-keys"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210003    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210009    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210018    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210023    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210031    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" containerName="payload-processing"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210036    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" containerName="payload-processing"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210044    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb2647a-5179-4b44-bda7-59962133486e" containerName="payload-pre-processing"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210049    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb2647a-5179-4b44-bda7-59962133486e" containerName="payload-pre-processing"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210055    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe172484-05d1-4e64-8f82-10e10a3faa05" containerName="istio-proxy"
Sep 19 12:46:46.210104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210060    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe172484-05d1-4e64-8f82-10e10a3faa05" containerName="istio-proxy"
Sep 19 12:46:46.210478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210119    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e17c340-8041-4efe-8e2a-40b637cfe715" containerName="revoke-keys"
Sep 19 12:46:46.210478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210129    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9e3f6ee-b6ec-4899-897a-b81eab56dc3f" containerName="payload-processing"
Sep 19 12:46:46.210478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210135    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:46.210478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210145    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb2647a-5179-4b44-bda7-59962133486e" containerName="payload-pre-processing"
Sep 19 12:46:46.210478 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.210155    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe172484-05d1-4e64-8f82-10e10a3faa05" containerName="istio-proxy"
Sep 19 12:46:46.214757 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.214737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.217212 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.217190    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b0e30e-openshift-default-dockercfg-6wnx2\""
Sep 19 12:46:46.226008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.225980    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 12:46:46.358401 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358419    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358443    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358742 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358742 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358621    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358742 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr959\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.358742 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.358710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460046    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460140 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460094    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460169    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr959\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460830 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460635    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460830 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460769    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460830 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460804    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460861    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.460951 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.460914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.462640 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.462615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.462788 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.462772    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.469110 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.469087    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.469365 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.469347    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr959\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959\") pod \"e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") " pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.527137 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.527100    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:46.810013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.809989    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 12:46:46.812752 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:46:46.812718    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb361b4d4_bb4a_47e5_83a5_bd64cf0f6390.slice/crio-da4c4a56cccb77c67667876eb779911e7ca05bd9764a4bc142869aad5efbebc1 WatchSource:0}: Error finding container da4c4a56cccb77c67667876eb779911e7ca05bd9764a4bc142869aad5efbebc1: Status 404 returned error can't find the container with id da4c4a56cccb77c67667876eb779911e7ca05bd9764a4bc142869aad5efbebc1
Sep 19 12:46:46.815112 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.815084    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:46:46.815172 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.815150    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:46:46.815213 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:46.815177    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:46:47.017645 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:47.017612    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerStarted","Data":"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"}
Sep 19 12:46:47.017818 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:47.017664    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerStarted","Data":"da4c4a56cccb77c67667876eb779911e7ca05bd9764a4bc142869aad5efbebc1"}
Sep 19 12:46:47.042079 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:47.042027    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podStartSLOduration=1.04201065 podStartE2EDuration="1.04201065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:47.040727817 +0000 UTC m=+1291.330283588" watchObservedRunningTime="2026-09-19 12:46:47.04201065 +0000 UTC m=+1291.331566419"
Sep 19 12:46:47.527660 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:47.527624    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:47.533171 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:47.533146    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:48.023560 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:48.023535    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:48.024511 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:48.024470    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:46:53.365799 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.365765    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 12:46:53.366219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.366205    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e30b02-fbee-40fb-aa51-ba48f1e1acf6" containerName="istio-proxy"
Sep 19 12:46:53.369567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.369551    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.371782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.371755    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-b0e30e-dockercfg-btdpn\""
Sep 19 12:46:53.372070 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.372054    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-b0e30e\""
Sep 19 12:46:53.378996 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.378970    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 12:46:53.401588 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.401547    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 12:46:53.405127 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.405105    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.411081 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.411054    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 12:46:53.528938 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.528904    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r69h4\" (UniqueName: \"kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.529125 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.528973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.529125 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.529036    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.529125 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.529098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxlf\" (UniqueName: \"kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.630347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.630310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxlf\" (UniqueName: \"kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.630547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.630368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r69h4\" (UniqueName: \"kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.630547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.630416    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.630547 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.630479    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.631174 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.631153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.631245 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.631186    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.639197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.639170    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxlf\" (UniqueName: \"kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf\") pod \"payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.639360 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.639339    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r69h4\" (UniqueName: \"kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4\") pod \"payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.681237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.681210    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:53.717451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.717422    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:53.970544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:53.968985    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 12:46:54.002628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.002606    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 12:46:54.004732 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:46:54.004705    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda868abf8_beb6_4785_b51b_0701b4628feb.slice/crio-c2ffa25ac50fe15f969a7ff686be07760725fc43b383564bb34d43c5e741e153 WatchSource:0}: Error finding container c2ffa25ac50fe15f969a7ff686be07760725fc43b383564bb34d43c5e741e153: Status 404 returned error can't find the container with id c2ffa25ac50fe15f969a7ff686be07760725fc43b383564bb34d43c5e741e153
Sep 19 12:46:54.058674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.058396    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" event={"ID":"414a4cfb-0671-4081-b387-ddf45268f021","Type":"ContainerStarted","Data":"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"}
Sep 19 12:46:54.058674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.058467    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" event={"ID":"414a4cfb-0671-4081-b387-ddf45268f021","Type":"ContainerStarted","Data":"c8b2904ad384c55e6824f9908a5171acf3ac9ce76905874855445de333945ec7"}
Sep 19 12:46:54.058674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.058639    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:46:54.061096 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.061063    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" event={"ID":"a868abf8-beb6-4785-b51b-0701b4628feb","Type":"ContainerStarted","Data":"c2ffa25ac50fe15f969a7ff686be07760725fc43b383564bb34d43c5e741e153"}
Sep 19 12:46:54.061216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.061200    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:46:54.077861 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.077809    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" podStartSLOduration=1.07779291 podStartE2EDuration="1.07779291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:54.075183778 +0000 UTC m=+1298.364739546" watchObservedRunningTime="2026-09-19 12:46:54.07779291 +0000 UTC m=+1298.367348679"
Sep 19 12:46:54.093275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:54.093225    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" podStartSLOduration=1.093207144 podStartE2EDuration="1.093207144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:54.091472882 +0000 UTC m=+1298.381028651" watchObservedRunningTime="2026-09-19 12:46:54.093207144 +0000 UTC m=+1298.382762913"
Sep 19 12:46:55.069562 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:46:55.069524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" event={"ID":"a868abf8-beb6-4785-b51b-0701b4628feb","Type":"ContainerStarted","Data":"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"}
Sep 19 12:47:05.072744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:05.072708    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 12:47:06.078775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:06.078732    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 12:47:12.170440 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.170405    2568 generic.go:356] "Generic (PLEG): container finished" podID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerID="63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0" exitCode=0
Sep 19 12:47:12.170885 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.170478    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerDied","Data":"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"}
Sep 19 12:47:12.171030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.171014    2568 scope.go:117] "RemoveContainer" containerID="63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"
Sep 19 12:47:12.172184 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.172156    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:12.172247 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.172221    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:12.172283 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:12.172251    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:13.182065 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:13.182013    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerStarted","Data":"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"}
Sep 19 12:47:13.527763 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:13.527657    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:47:13.529132 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:13.529103    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:47:13.529267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:13.529169    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 19 12:47:14.528209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:14.528174    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:47:14.528623 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:14.528227    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 19 12:47:15.528313 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:15.528278    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:47:15.528748 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:15.528331    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 19 12:47:16.528003 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:16.527963    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:47:16.528193 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:16.528020    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 19 12:47:17.527990 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:17.527953    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:47:17.528399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:17.528011    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 19 12:47:17.858340 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:17.858247    2568 scope.go:117] "RemoveContainer" containerID="d7b45aa094f250dbe1130ecc6d3ab2bd4a9e707a3f4be7984197c808793130fb"
Sep 19 12:47:17.886663 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:17.886640    2568 scope.go:117] "RemoveContainer" containerID="dbb8322b709dad5fd22fd7a1e35cd88d3d2b43fc6c08f0620b45d56b02e22348"
Sep 19 12:47:18.024332 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:18.024293    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:47:18.532659 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:18.532630    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:47:18.533717 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:18.533698    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 12:47:19.505662 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.505631    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 12:47:19.511149 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.511129    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.513483 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.513459    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-5d6124-openshift-default-dockercfg-jllm7\""
Sep 19 12:47:19.522943 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.522920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606298    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606382    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606604    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nklt\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606700    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.607050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.606735    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708068 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708024    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708084    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nklt\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708193    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708282 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708255    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708314    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.708629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.708426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.709848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.709790    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.710196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.710155    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.710347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.710324    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.710574 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.710549    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.711656 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.710705    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.713743 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.712582    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.713863 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.713796    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.717773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.717754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.717891 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.717827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nklt\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt\") pod \"e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") " pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:19.823532 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:19.823436    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:20.100271 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.100249    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 12:47:20.102946 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:47:20.102918    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ae1836_8a3a_46ff_8f37_6bf2ad5182dd.slice/crio-88117753b2af7dd00bca06e9c9112b7087aa0e65de2f1f55753474b5fb3bcf1d WatchSource:0}: Error finding container 88117753b2af7dd00bca06e9c9112b7087aa0e65de2f1f55753474b5fb3bcf1d: Status 404 returned error can't find the container with id 88117753b2af7dd00bca06e9c9112b7087aa0e65de2f1f55753474b5fb3bcf1d
Sep 19 12:47:20.105406 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.105372    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:20.105534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.105450    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:20.105534 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.105507    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:20.233487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.233450    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerStarted","Data":"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"}
Sep 19 12:47:20.233671 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.233533    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerStarted","Data":"88117753b2af7dd00bca06e9c9112b7087aa0e65de2f1f55753474b5fb3bcf1d"}
Sep 19 12:47:20.258267 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.258217    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podStartSLOduration=1.258200591 podStartE2EDuration="1.258200591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:47:20.255328129 +0000 UTC m=+1324.544883904" watchObservedRunningTime="2026-09-19 12:47:20.258200591 +0000 UTC m=+1324.547756360"
Sep 19 12:47:20.824444 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:20.824407    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:21.830134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:21.830102    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:22.245256 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:22.245222    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:22.246396 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:22.246372    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:27.440234 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.440196    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 12:47:27.444103 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.444087    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.446776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.446751    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-5d6124-dockercfg-qscqr\""
Sep 19 12:47:27.446883 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.446751    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-5d6124\""
Sep 19 12:47:27.453023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.453003    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 12:47:27.591567 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.591525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dzk7\" (UniqueName: \"kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.591739 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.591722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.693238 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.693127    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.693238 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.693199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dzk7\" (UniqueName: \"kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.693918 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.693896    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.701199 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.701180    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dzk7\" (UniqueName: \"kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7\") pod \"payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:27.756622 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:27.756588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:28.033749 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:28.033725    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 12:47:28.035367 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:47:28.035334    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c1f288_ac78_4adc_beb4_1d7605102ede.slice/crio-37dcc3c3c60d8e4bfb90e4f06e262c5b24a0bcc6fa6d537dc18fd987b9185ad1 WatchSource:0}: Error finding container 37dcc3c3c60d8e4bfb90e4f06e262c5b24a0bcc6fa6d537dc18fd987b9185ad1: Status 404 returned error can't find the container with id 37dcc3c3c60d8e4bfb90e4f06e262c5b24a0bcc6fa6d537dc18fd987b9185ad1
Sep 19 12:47:28.283388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:28.283291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" event={"ID":"77c1f288-ac78-4adc-beb4-1d7605102ede","Type":"ContainerStarted","Data":"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"}
Sep 19 12:47:28.283388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:28.283353    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" event={"ID":"77c1f288-ac78-4adc-beb4-1d7605102ede","Type":"ContainerStarted","Data":"37dcc3c3c60d8e4bfb90e4f06e262c5b24a0bcc6fa6d537dc18fd987b9185ad1"}
Sep 19 12:47:28.283621 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:28.283521    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:28.300968 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:28.300919    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" podStartSLOduration=1.300899111 podStartE2EDuration="1.300899111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:47:28.298628596 +0000 UTC m=+1332.588184365" watchObservedRunningTime="2026-09-19 12:47:28.300899111 +0000 UTC m=+1332.590454880"
Sep 19 12:47:39.294735 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:39.294653    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 12:47:52.247530 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.247471    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:52.247992 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.247558    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:52.434865 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.434830    2568 generic.go:356] "Generic (PLEG): container finished" podID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerID="30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729" exitCode=0
Sep 19 12:47:52.435036 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.434872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerDied","Data":"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"}
Sep 19 12:47:52.435387 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.435369    2568 scope.go:117] "RemoveContainer" containerID="30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"
Sep 19 12:47:52.436650 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.436620    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:52.436715 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.436689    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:52.436752 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:52.436722    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:47:53.443106 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:53.443066    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerStarted","Data":"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"}
Sep 19 12:47:53.824573 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:53.824459    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:47:53.824775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:53.824750    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:53.824869 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:53.824807    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:54.824236 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:54.824204    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:54.824627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:54.824274    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:55.824066 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:55.824032    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:55.824248 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:55.824099    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:56.824931 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:56.824894    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:56.825325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:56.824950    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:57.824583 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:57.824542    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:57.824779 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:57.824616    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:58.824604 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:58.824557    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:58.825009 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:58.824628    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:47:59.824325 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:59.824295    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 12:47:59.824523 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:47:59.824366    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 12:48:00.828490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:00.828459    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:48:00.829013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:00.828993    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:48:00.829592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:00.829572    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 12:48:11.211481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.211449    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"]
Sep 19 12:48:11.215158 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.215139    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:11.217636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.217616    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:48:11.217730 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.217632    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:48:11.221787 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.221766    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"]
Sep 19 12:48:11.264762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.264726    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c\" (UID: \"e4b4426f-9b39-4303-9970-00c2f8f06806\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:11.365670 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.365630    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c\" (UID: \"e4b4426f-9b39-4303-9970-00c2f8f06806\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:11.366278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.366247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c\" (UID: \"e4b4426f-9b39-4303-9970-00c2f8f06806\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:11.527183 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.527092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:11.798214 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:11.798188    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"]
Sep 19 12:48:11.800660 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:48:11.800630    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b4426f_9b39_4303_9970_00c2f8f06806.slice/crio-85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359 WatchSource:0}: Error finding container 85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359: Status 404 returned error can't find the container with id 85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359
Sep 19 12:48:12.560111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:12.560079    2568 generic.go:356] "Generic (PLEG): container finished" podID="e4b4426f-9b39-4303-9970-00c2f8f06806" containerID="8fd060990abcb75e541411e48e2952acf47b50948b337311716f6bf2e485513c" exitCode=0
Sep 19 12:48:12.560487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:12.560120    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c" event={"ID":"e4b4426f-9b39-4303-9970-00c2f8f06806","Type":"ContainerDied","Data":"8fd060990abcb75e541411e48e2952acf47b50948b337311716f6bf2e485513c"}
Sep 19 12:48:12.560487 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:12.560172    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c" event={"ID":"e4b4426f-9b39-4303-9970-00c2f8f06806","Type":"ContainerStarted","Data":"85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359"}
Sep 19 12:48:13.704368 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:13.704343    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:13.788293 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:13.788262    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca\") pod \"e4b4426f-9b39-4303-9970-00c2f8f06806\" (UID: \"e4b4426f-9b39-4303-9970-00c2f8f06806\") "
Sep 19 12:48:13.788631 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:13.788610    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4b4426f-9b39-4303-9970-00c2f8f06806" (UID: "e4b4426f-9b39-4303-9970-00c2f8f06806"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:48:13.889769 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:13.889743    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4b4426f-9b39-4303-9970-00c2f8f06806-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:48:14.240388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.240310    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:48:14.240784 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.240771    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4b4426f-9b39-4303-9970-00c2f8f06806" containerName="revoke-keys"
Sep 19 12:48:14.240833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.240787    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4b4426f-9b39-4303-9970-00c2f8f06806" containerName="revoke-keys"
Sep 19 12:48:14.240870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.240855    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4b4426f-9b39-4303-9970-00c2f8f06806" containerName="revoke-keys"
Sep 19 12:48:14.244056 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.244039    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.247226 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.247204    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"openshift-service-ca.crt\""
Sep 19 12:48:14.247752 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.247733    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"default-dockercfg-gmwjx\""
Sep 19 12:48:14.247824 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.247737    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"kube-root-ca.crt\""
Sep 19 12:48:14.247895 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.247879    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"auth-test-model-b0e30e-kserve-self-signed-certs\""
Sep 19 12:48:14.257484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.257460    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:48:14.294580 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294550    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkf7v\" (UniqueName: \"kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.294746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294597    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.294746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.294746 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.294854 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.294854 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.294794    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.396897 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.396859    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.397219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.397551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397528    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkf7v\" (UniqueName: \"kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.397744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.397883 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.398012 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.398681 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.398644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.399036 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.397401    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.399322 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.399292    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.401885 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.401844    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.402218 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.402168    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.416778 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.408855    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkf7v\" (UniqueName: \"kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v\") pod \"auth-test-model-b0e30e-kserve-f4797cbc-ws7xd\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.553322 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.553229    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:14.574024 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.574001    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"
Sep 19 12:48:14.574158 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.573994    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c" event={"ID":"e4b4426f-9b39-4303-9970-00c2f8f06806","Type":"ContainerDied","Data":"85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359"}
Sep 19 12:48:14.574158 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.574128    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85b306c2991f243a7b470f92faf1a8aa2a3b98468d621e268fc6f90d9dbab359"
Sep 19 12:48:14.829554 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:14.829527    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:48:14.831575 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:48:14.831543    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97162928_fdf3_4c9f_9321_846e87911a3d.slice/crio-8ee9b08faac2fd8bdf76b2d90272380ff32f15f94d448aa21fb3f42c2c862f9f WatchSource:0}: Error finding container 8ee9b08faac2fd8bdf76b2d90272380ff32f15f94d448aa21fb3f42c2c862f9f: Status 404 returned error can't find the container with id 8ee9b08faac2fd8bdf76b2d90272380ff32f15f94d448aa21fb3f42c2c862f9f
Sep 19 12:48:15.580926 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:15.580891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" event={"ID":"97162928-fdf3-4c9f-9321-846e87911a3d","Type":"ContainerStarted","Data":"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"}
Sep 19 12:48:15.581110 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:15.580936    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" event={"ID":"97162928-fdf3-4c9f-9321-846e87911a3d","Type":"ContainerStarted","Data":"8ee9b08faac2fd8bdf76b2d90272380ff32f15f94d448aa21fb3f42c2c862f9f"}
Sep 19 12:48:15.602260 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:15.602210    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" podStartSLOduration=1.602194332 podStartE2EDuration="1.602194332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:48:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:48:15.600229463 +0000 UTC m=+1379.889785233" watchObservedRunningTime="2026-09-19 12:48:15.602194332 +0000 UTC m=+1379.891750100"
Sep 19 12:48:24.554071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:24.554029    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:24.554673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:24.554083    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:24.566848 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:24.566824    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:24.645386 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:24.645348    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:48:26.909008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:26.908973    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:48:26.909491 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:26.909251    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" podUID="28c1af05-f903-4a59-a94c-35c2f4179a78" containerName="payload-processing" containerID="cri-o://fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595" gracePeriod=30
Sep 19 12:48:26.917380 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:26.917346    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:48:26.917948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:26.917623    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" podUID="dade702b-d547-4fe3-9ee2-29e1c557c278" containerName="payload-pre-processing" containerID="cri-o://d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500" gracePeriod=30
Sep 19 12:48:27.204623 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.204525    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"]
Sep 19 12:48:27.218967 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.218936    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9a75910-lhv6c"]
Sep 19 12:48:27.240533 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.240507    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:48:27.243759 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.243736    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:48:27.328795 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.328758    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume\") pod \"dade702b-d547-4fe3-9ee2-29e1c557c278\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") "
Sep 19 12:48:27.328995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.328900    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cmkp\" (UniqueName: \"kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp\") pod \"28c1af05-f903-4a59-a94c-35c2f4179a78\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") "
Sep 19 12:48:27.328995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.328961    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zsqh\" (UniqueName: \"kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh\") pod \"dade702b-d547-4fe3-9ee2-29e1c557c278\" (UID: \"dade702b-d547-4fe3-9ee2-29e1c557c278\") "
Sep 19 12:48:27.329107 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.328996    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume\") pod \"28c1af05-f903-4a59-a94c-35c2f4179a78\" (UID: \"28c1af05-f903-4a59-a94c-35c2f4179a78\") "
Sep 19 12:48:27.329158 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.329123    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dade702b-d547-4fe3-9ee2-29e1c557c278" (UID: "dade702b-d547-4fe3-9ee2-29e1c557c278"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:48:27.329377 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.329351    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dade702b-d547-4fe3-9ee2-29e1c557c278-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:48:27.329451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.329386    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28c1af05-f903-4a59-a94c-35c2f4179a78" (UID: "28c1af05-f903-4a59-a94c-35c2f4179a78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:48:27.331066 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.331036    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh" (OuterVolumeSpecName: "kube-api-access-9zsqh") pod "dade702b-d547-4fe3-9ee2-29e1c557c278" (UID: "dade702b-d547-4fe3-9ee2-29e1c557c278"). InnerVolumeSpecName "kube-api-access-9zsqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:48:27.331211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.331114    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp" (OuterVolumeSpecName: "kube-api-access-8cmkp") pod "28c1af05-f903-4a59-a94c-35c2f4179a78" (UID: "28c1af05-f903-4a59-a94c-35c2f4179a78"). InnerVolumeSpecName "kube-api-access-8cmkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:48:27.430633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.430594    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cmkp\" (UniqueName: \"kubernetes.io/projected/28c1af05-f903-4a59-a94c-35c2f4179a78-kube-api-access-8cmkp\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:48:27.430633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.430625    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zsqh\" (UniqueName: \"kubernetes.io/projected/dade702b-d547-4fe3-9ee2-29e1c557c278-kube-api-access-9zsqh\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:48:27.430633 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.430634    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c1af05-f903-4a59-a94c-35c2f4179a78-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:48:27.651608 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.651575    2568 generic.go:356] "Generic (PLEG): container finished" podID="28c1af05-f903-4a59-a94c-35c2f4179a78" containerID="fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595" exitCode=0
Sep 19 12:48:27.651793 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.651617    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"
Sep 19 12:48:27.651793 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.651662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" event={"ID":"28c1af05-f903-4a59-a94c-35c2f4179a78","Type":"ContainerDied","Data":"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"}
Sep 19 12:48:27.651793 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.651718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch" event={"ID":"28c1af05-f903-4a59-a94c-35c2f4179a78","Type":"ContainerDied","Data":"ffad3e6b748fab4214a09e6b2deda01bbe0957880e6fb7289868c2c712126c8b"}
Sep 19 12:48:27.651793 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.651751    2568 scope.go:117] "RemoveContainer" containerID="fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"
Sep 19 12:48:27.653025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.653006    2568 generic.go:356] "Generic (PLEG): container finished" podID="dade702b-d547-4fe3-9ee2-29e1c557c278" containerID="d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500" exitCode=0
Sep 19 12:48:27.653104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.653093    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"
Sep 19 12:48:27.653161 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.653129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" event={"ID":"dade702b-d547-4fe3-9ee2-29e1c557c278","Type":"ContainerDied","Data":"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"}
Sep 19 12:48:27.653219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.653183    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq" event={"ID":"dade702b-d547-4fe3-9ee2-29e1c557c278","Type":"ContainerDied","Data":"0a0becc96616a3a97504828cb493d2747ae162d907745f49f7c7ee89df13d5bc"}
Sep 19 12:48:27.660969 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.660957    2568 scope.go:117] "RemoveContainer" containerID="fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"
Sep 19 12:48:27.661240 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:48:27.661221    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595\": container with ID starting with fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595 not found: ID does not exist" containerID="fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"
Sep 19 12:48:27.661312 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.661247    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595"} err="failed to get container status \"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595\": rpc error: code = NotFound desc = could not find container \"fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595\": container with ID starting with fab6eeddf5d561cec043722c4c6f6d9f7482766584c7d00861e1ca7f2a5ce595 not found: ID does not exist"
Sep 19 12:48:27.661312 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.661267    2568 scope.go:117] "RemoveContainer" containerID="d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"
Sep 19 12:48:27.669740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.669724    2568 scope.go:117] "RemoveContainer" containerID="d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"
Sep 19 12:48:27.670011 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:48:27.669992    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500\": container with ID starting with d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500 not found: ID does not exist" containerID="d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"
Sep 19 12:48:27.670063 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.670016    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500"} err="failed to get container status \"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500\": rpc error: code = NotFound desc = could not find container \"d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500\": container with ID starting with d9004be76454e931eea875b21ca599ed0503f14a03dc85abe7200045aaa0f500 not found: ID does not exist"
Sep 19 12:48:27.678764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.678741    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:48:27.684111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.684092    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9a75910-7cd8d8847c-v4qch"]
Sep 19 12:48:27.694470 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.694449    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:48:27.696408 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.696388    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9a75910-68b68b5466-nkssq"]
Sep 19 12:48:27.724054 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.724030    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c1af05-f903-4a59-a94c-35c2f4179a78" path="/var/lib/kubelet/pods/28c1af05-f903-4a59-a94c-35c2f4179a78/volumes"
Sep 19 12:48:27.724401 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.724389    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dade702b-d547-4fe3-9ee2-29e1c557c278" path="/var/lib/kubelet/pods/dade702b-d547-4fe3-9ee2-29e1c557c278/volumes"
Sep 19 12:48:27.724702 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:48:27.724690    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4b4426f-9b39-4303-9970-00c2f8f06806" path="/var/lib/kubelet/pods/e4b4426f-9b39-4303-9970-00c2f8f06806/volumes"
Sep 19 12:49:50.225653 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:50.225615    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:49:50.226259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:50.225880    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy" containerID="cri-o://0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c" gracePeriod=30
Sep 19 12:49:55.474642 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.474613    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:49:55.521317 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521237    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521317 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521312    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521344    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521405    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521462    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cds87\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521491    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521546    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521619 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521583    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data\") pod \"3814fca7-0020-47c1-992f-bb74da538c10\" (UID: \"3814fca7-0020-47c1-992f-bb74da538c10\") "
Sep 19 12:49:55.521916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521721    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:49:55.521916 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.521829    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:49:55.522063 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.522034    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:49:55.522123 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.522077    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.522123 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.522095    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3814fca7-0020-47c1-992f-bb74da538c10-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.522338 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.522310    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data" (OuterVolumeSpecName: "istio-data") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:49:55.522415 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.522372    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:49:55.523717 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.523691    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:49:55.524021 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.523993    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token" (OuterVolumeSpecName: "istio-token") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:49:55.524484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.524460    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:49:55.524600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.524466    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87" (OuterVolumeSpecName: "kube-api-access-cds87") pod "3814fca7-0020-47c1-992f-bb74da538c10" (UID: "3814fca7-0020-47c1-992f-bb74da538c10"). InnerVolumeSpecName "kube-api-access-cds87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:49:55.622657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622628    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cds87\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-kube-api-access-cds87\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622654    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622657 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622663    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622672    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622681    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3814fca7-0020-47c1-992f-bb74da538c10-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622688    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3814fca7-0020-47c1-992f-bb74da538c10-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:55.622896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:55.622695    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3814fca7-0020-47c1-992f-bb74da538c10-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:49:56.148286 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.148252    2568 generic.go:356] "Generic (PLEG): container finished" podID="3814fca7-0020-47c1-992f-bb74da538c10" containerID="0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c" exitCode=0
Sep 19 12:49:56.148490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.148309    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"
Sep 19 12:49:56.148490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.148336    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerDied","Data":"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"}
Sep 19 12:49:56.148490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.148383    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm" event={"ID":"3814fca7-0020-47c1-992f-bb74da538c10","Type":"ContainerDied","Data":"3fdf1b8e17c00cacbfabeab42f84ef311b21e3443e6ddcd64e51890916700d20"}
Sep 19 12:49:56.148490 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.148405    2568 scope.go:117] "RemoveContainer" containerID="0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"
Sep 19 12:49:56.157829 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.157813    2568 scope.go:117] "RemoveContainer" containerID="8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"
Sep 19 12:49:56.165729 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.165711    2568 scope.go:117] "RemoveContainer" containerID="0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"
Sep 19 12:49:56.166666 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:49:56.166626    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c\": container with ID starting with 0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c not found: ID does not exist" containerID="0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"
Sep 19 12:49:56.166788 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.166676    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c"} err="failed to get container status \"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c\": rpc error: code = NotFound desc = could not find container \"0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c\": container with ID starting with 0d3ed7fd7d00c7a186248db55961b272f2f1afa1484c36dcb0eca87298fdb23c not found: ID does not exist"
Sep 19 12:49:56.166788 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.166700    2568 scope.go:117] "RemoveContainer" containerID="8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"
Sep 19 12:49:56.166982 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:49:56.166961    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f\": container with ID starting with 8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f not found: ID does not exist" containerID="8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"
Sep 19 12:49:56.167026 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.166988    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f"} err="failed to get container status \"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f\": rpc error: code = NotFound desc = could not find container \"8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f\": container with ID starting with 8246b61d8bdd44322cf82985988498a842a1506a74cc9a6e70ac8db6fe8e241f not found: ID does not exist"
Sep 19 12:49:56.169153 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.169136    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:49:56.172287 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:56.172267    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9a75910-openshift-default-5689b94cb8-6jnqm"]
Sep 19 12:49:57.723209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:57.723178    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3814fca7-0020-47c1-992f-bb74da538c10" path="/var/lib/kubelet/pods/3814fca7-0020-47c1-992f-bb74da538c10/volumes"
Sep 19 12:49:59.840462 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.840425    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:49:59.841084 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841065    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:49:59.841142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841088    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:49:59.841142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841124    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dade702b-d547-4fe3-9ee2-29e1c557c278" containerName="payload-pre-processing"
Sep 19 12:49:59.841142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841133    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dade702b-d547-4fe3-9ee2-29e1c557c278" containerName="payload-pre-processing"
Sep 19 12:49:59.841241 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841146    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c1af05-f903-4a59-a94c-35c2f4179a78" containerName="payload-processing"
Sep 19 12:49:59.841241 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841155    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c1af05-f903-4a59-a94c-35c2f4179a78" containerName="payload-processing"
Sep 19 12:49:59.841315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841251    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:49:59.841315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841263    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c1af05-f903-4a59-a94c-35c2f4179a78" containerName="payload-processing"
Sep 19 12:49:59.841315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841276    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dade702b-d547-4fe3-9ee2-29e1c557c278" containerName="payload-pre-processing"
Sep 19 12:49:59.841315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.841287    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:49:59.846426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.846403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.849860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.849835    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"kube-root-ca.crt\""
Sep 19 12:49:59.849860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.849852    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"embedding-w0-a2e924-kserve-self-signed-certs\""
Sep 19 12:49:59.850038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.849881    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"openshift-service-ca.crt\""
Sep 19 12:49:59.850038 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.849835    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"default-dockercfg-7btsc\""
Sep 19 12:49:59.854481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.854397    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:49:59.968206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjj7v\" (UniqueName: \"kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.968206 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968219    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.968424 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968238    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.968424 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968257    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.968424 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968327    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:49:59.968553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:49:59.968464    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069403 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjj7v\" (UniqueName: \"kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069601 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069768 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069604    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069819 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069800    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069881 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069863    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.069942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.069928    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.071857 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.071832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.072030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.072014    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.077276 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.077256    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjj7v\" (UniqueName: \"kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v\") pod \"embedding-w0-a2e924-kserve-546f994fbf-2v5p2\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") " pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.157622 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.157588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:00.442051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:00.442026    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:50:00.443998 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:50:00.443972    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd15d36ab_2f40_4a68_9937_9b5c1449e7fe.slice/crio-14d9f5facc37a69892af1650aace5227bd4b508c5e39e387dbf0c102b7fdc646 WatchSource:0}: Error finding container 14d9f5facc37a69892af1650aace5227bd4b508c5e39e387dbf0c102b7fdc646: Status 404 returned error can't find the container with id 14d9f5facc37a69892af1650aace5227bd4b508c5e39e387dbf0c102b7fdc646
Sep 19 12:50:01.179789 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:01.179751    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" event={"ID":"d15d36ab-2f40-4a68-9937-9b5c1449e7fe","Type":"ContainerStarted","Data":"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"}
Sep 19 12:50:01.179789 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:01.179794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" event={"ID":"d15d36ab-2f40-4a68-9937-9b5c1449e7fe","Type":"ContainerStarted","Data":"14d9f5facc37a69892af1650aace5227bd4b508c5e39e387dbf0c102b7fdc646"}
Sep 19 12:50:01.198228 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:01.198178    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" podStartSLOduration=2.198163108 podStartE2EDuration="2.198163108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:50:01.196595753 +0000 UTC m=+1485.486151521" watchObservedRunningTime="2026-09-19 12:50:01.198163108 +0000 UTC m=+1485.487718876"
Sep 19 12:50:10.158033 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:10.157996    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:10.158436 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:10.158045    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:10.170298 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:10.170275    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:10.239219 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:10.239188    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:50:17.742977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:17.742940    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:50:17.744683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:17.744661    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:50:17.963963 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:17.963935    2568 scope.go:117] "RemoveContainer" containerID="09ed97316a867b5e7d02406645b6661669445c5ca2c0d0666b5e67113e3cd1b3"
Sep 19 12:50:17.972367 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:17.972339    2568 scope.go:117] "RemoveContainer" containerID="1462ff5f6509edeaab3d4eaded9ebc966af1e946d9dba2453b9780aed91e9fc0"
Sep 19 12:50:44.074414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.074340    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:50:44.075030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.075005    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:50:44.075030 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.075030    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3814fca7-0020-47c1-992f-bb74da538c10" containerName="istio-proxy"
Sep 19 12:50:44.078110 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.078090    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.080477 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.080458    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-49e81a-openshift-default-dockercfg-tgnt8\""
Sep 19 12:50:44.093278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.093252    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:50:44.210429 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210389    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210472    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210637    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210694    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ngb\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.210809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.210737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.311955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.311922    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.311965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.311986    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312013    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312056    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ngb\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312338    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312597    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312795 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312665    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312795 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312683    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.312899 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.312794    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.314434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.314410    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.314591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.314576    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.319869 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.319852    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.320064 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.320047    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ngb\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb\") pod \"e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") " pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.388454 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.388401    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:44.673923 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.673894    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:50:44.675518 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:50:44.675466    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd095bf6_f086_4215_b50e_e12b8fe81e27.slice/crio-b1b1a0889eb8a960ebd3cd6650d5d0d5d88b3b037b80e8d2e138d5029d2acfa0 WatchSource:0}: Error finding container b1b1a0889eb8a960ebd3cd6650d5d0d5d88b3b037b80e8d2e138d5029d2acfa0: Status 404 returned error can't find the container with id b1b1a0889eb8a960ebd3cd6650d5d0d5d88b3b037b80e8d2e138d5029d2acfa0
Sep 19 12:50:44.677955 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.677914    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:50:44.678034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.678000    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:50:44.678070 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:44.678040    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:50:45.423771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:45.423737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerStarted","Data":"d6e6bbcdea8db6007ae5915b5021402c41e41d67150aa7022b2834c348928a3c"}
Sep 19 12:50:45.424160 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:45.423789    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerStarted","Data":"b1b1a0889eb8a960ebd3cd6650d5d0d5d88b3b037b80e8d2e138d5029d2acfa0"}
Sep 19 12:50:45.450209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:45.450161    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podStartSLOduration=1.450147197 podStartE2EDuration="1.450147197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:50:45.448269295 +0000 UTC m=+1529.737825076" watchObservedRunningTime="2026-09-19 12:50:45.450147197 +0000 UTC m=+1529.739702968"
Sep 19 12:50:46.389428 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:46.389394    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:46.394740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:46.394713    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:46.429881 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:46.429848    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:46.430831 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:46.430814    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:50:52.898259 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:52.898223    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:50:52.902020 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:52.902001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:52.904469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:52.904443    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-49e81a\""
Sep 19 12:50:52.912168 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:52.912145    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:50:53.000920 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.000886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wp7\" (UniqueName: \"kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.001096 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.000939    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.102318 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.102282    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.102549 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.102398    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wp7\" (UniqueName: \"kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.103019 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.102999    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.111045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.111023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wp7\" (UniqueName: \"kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7\") pod \"payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.214561 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.214464    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:53.485334 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:53.485311    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:50:53.487403 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:50:53.487379    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced69de1_1d5f_4038_a8f0_b58376ace694.slice/crio-f928d155ad38c98415cbb978ac6bff3532ffbd7ba0964942dbe83ac7c54e0972 WatchSource:0}: Error finding container f928d155ad38c98415cbb978ac6bff3532ffbd7ba0964942dbe83ac7c54e0972: Status 404 returned error can't find the container with id f928d155ad38c98415cbb978ac6bff3532ffbd7ba0964942dbe83ac7c54e0972
Sep 19 12:50:54.473315 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:54.473263    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" event={"ID":"ced69de1-1d5f-4038-a8f0-b58376ace694","Type":"ContainerStarted","Data":"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"}
Sep 19 12:50:54.473777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:54.473364    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:50:54.473777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:54.473383    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" event={"ID":"ced69de1-1d5f-4038-a8f0-b58376ace694","Type":"ContainerStarted","Data":"f928d155ad38c98415cbb978ac6bff3532ffbd7ba0964942dbe83ac7c54e0972"}
Sep 19 12:50:54.489359 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:50:54.489300    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" podStartSLOduration=2.489281587 podStartE2EDuration="2.489281587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:50:54.488288795 +0000 UTC m=+1538.777844566" watchObservedRunningTime="2026-09-19 12:50:54.489281587 +0000 UTC m=+1538.778837359"
Sep 19 12:51:05.482518 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:05.482460    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:51:11.871189 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.871150    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"]
Sep 19 12:51:11.875007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.874986    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:11.877422 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.877384    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:51:11.877422 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.877384    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:51:11.881880 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.881859    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"]
Sep 19 12:51:11.996122 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:11.996085    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn\" (UID: \"895058f3-caaa-49b2-8833-227dc26cec45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:12.097511 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:12.097473    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn\" (UID: \"895058f3-caaa-49b2-8833-227dc26cec45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:12.098334 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:12.098306    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn\" (UID: \"895058f3-caaa-49b2-8833-227dc26cec45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:12.188375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:12.188341    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:12.461126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:12.461096    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"]
Sep 19 12:51:12.462343 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:51:12.462317    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895058f3_caaa_49b2_8833_227dc26cec45.slice/crio-d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf WatchSource:0}: Error finding container d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf: Status 404 returned error can't find the container with id d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf
Sep 19 12:51:12.572998 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:12.572964    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn" event={"ID":"895058f3-caaa-49b2-8833-227dc26cec45","Type":"ContainerStarted","Data":"d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf"}
Sep 19 12:51:13.579569 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.579470    2568 generic.go:356] "Generic (PLEG): container finished" podID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerID="d6e6bbcdea8db6007ae5915b5021402c41e41d67150aa7022b2834c348928a3c" exitCode=0
Sep 19 12:51:13.579569 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.579544    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerDied","Data":"d6e6bbcdea8db6007ae5915b5021402c41e41d67150aa7022b2834c348928a3c"}
Sep 19 12:51:13.580157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.580008    2568 scope.go:117] "RemoveContainer" containerID="d6e6bbcdea8db6007ae5915b5021402c41e41d67150aa7022b2834c348928a3c"
Sep 19 12:51:13.581307 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.581272    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:13.581426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.581348    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:13.581426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.581373    2568 generic.go:356] "Generic (PLEG): container finished" podID="895058f3-caaa-49b2-8833-227dc26cec45" containerID="0491eb198c641a18447867b33021157820633098b93e8374dda3e231f80bb090" exitCode=0
Sep 19 12:51:13.581426 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.581390    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:13.581575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:13.581456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn" event={"ID":"895058f3-caaa-49b2-8833-227dc26cec45","Type":"ContainerDied","Data":"0491eb198c641a18447867b33021157820633098b93e8374dda3e231f80bb090"}
Sep 19 12:51:14.591445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:14.591412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerStarted","Data":"8727baff357f384c09d79324b05861e26deb8bf3d82ddb19c27196f9e6ccc6bb"}
Sep 19 12:51:14.765528 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:14.762231    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:14.829536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:14.829485    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca\") pod \"895058f3-caaa-49b2-8833-227dc26cec45\" (UID: \"895058f3-caaa-49b2-8833-227dc26cec45\") "
Sep 19 12:51:14.829820 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:14.829800    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "895058f3-caaa-49b2-8833-227dc26cec45" (UID: "895058f3-caaa-49b2-8833-227dc26cec45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:14.930792 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:14.930759    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895058f3-caaa-49b2-8833-227dc26cec45-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:15.389071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.389033    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:51:15.390385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.390345    2568 patch_prober.go:28] interesting pod/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 12:51:15.390598 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.390414    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 12:51:15.598591 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.598558    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn" event={"ID":"895058f3-caaa-49b2-8833-227dc26cec45","Type":"ContainerDied","Data":"d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf"}
Sep 19 12:51:15.598974 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.598597    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"
Sep 19 12:51:15.598974 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:15.598608    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5c2e130f3679a907ac459b99cbc5fb0853f206e5654269283755adb8cd45acf"
Sep 19 12:51:16.389841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:16.389803    2568 patch_prober.go:28] interesting pod/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 12:51:16.390023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:16.389867    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 12:51:16.430062 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:16.430024    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:51:16.762076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:16.761982    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:51:17.389903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:17.389862    2568 patch_prober.go:28] interesting pod/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 12:51:17.390092 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:17.389927    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 12:51:18.001145 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:18.001113    2568 scope.go:117] "RemoveContainer" containerID="effe0b34fae6dc8e4ec9ab9b8f72df860de7787d181f81c218abf3868a491a7f"
Sep 19 12:51:18.010275 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:18.010254    2568 scope.go:117] "RemoveContainer" containerID="9168b3ab547257dd84757d3859293487ad6e5daf39dd1104244cc91ff29c97ed"
Sep 19 12:51:18.389443 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:18.389394    2568 patch_prober.go:28] interesting pod/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 12:51:18.389628 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:18.389460    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 12:51:19.389903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:19.389857    2568 patch_prober.go:28] interesting pod/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 12:51:19.390295 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:19.389930    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 12:51:20.393157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:20.393129    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:51:20.394101 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:20.394077    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:51:23.877986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.877955    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:51:23.878389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.878374    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895058f3-caaa-49b2-8833-227dc26cec45" containerName="revoke-keys"
Sep 19 12:51:23.878434 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.878390    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="895058f3-caaa-49b2-8833-227dc26cec45" containerName="revoke-keys"
Sep 19 12:51:23.878481 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.878472    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="895058f3-caaa-49b2-8833-227dc26cec45" containerName="revoke-keys"
Sep 19 12:51:23.883226 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.883210    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:23.885960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.885940    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-513a51-dockercfg-njgv6\""
Sep 19 12:51:23.887225 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.887205    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-513a51\""
Sep 19 12:51:23.904051 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.904031    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:51:23.911775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.911750    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:51:23.915809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.915785    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:23.940264 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:23.940238    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:51:24.041685 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.041643    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2vkn\" (UniqueName: \"kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.041877 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.041759    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kvnk\" (UniqueName: \"kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.041877 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.041822    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.042010 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.041883    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.143372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.143281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2vkn\" (UniqueName: \"kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.143372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.143346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kvnk\" (UniqueName: \"kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.143372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.143372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.143632 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.143411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.144160 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.144141    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.144204 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.144145    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.151234 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.151210    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kvnk\" (UniqueName: \"kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk\") pod \"payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") " pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.151353 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.151279    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2vkn\" (UniqueName: \"kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn\") pod \"payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.193149 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.193111    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.225371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.225346    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.481179 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.481157    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:51:24.483251 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:51:24.483231    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0049e67f_330d_45a5_aea1_f77342590e19.slice/crio-15554fe5b2853af5ff3ec2645b90bacf0ddf6f86ea3eb973a128cdfc5ebf363f WatchSource:0}: Error finding container 15554fe5b2853af5ff3ec2645b90bacf0ddf6f86ea3eb973a128cdfc5ebf363f: Status 404 returned error can't find the container with id 15554fe5b2853af5ff3ec2645b90bacf0ddf6f86ea3eb973a128cdfc5ebf363f
Sep 19 12:51:24.525270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.525219    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:51:24.527440 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:51:24.527416    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f370df_beb3_49e8_809a_ffb218bba2c5.slice/crio-544cc362cd7468ce41678a9ad8bd3986ab3e7c88c3713f3a4d12f70ccfc4df8c WatchSource:0}: Error finding container 544cc362cd7468ce41678a9ad8bd3986ab3e7c88c3713f3a4d12f70ccfc4df8c: Status 404 returned error can't find the container with id 544cc362cd7468ce41678a9ad8bd3986ab3e7c88c3713f3a4d12f70ccfc4df8c
Sep 19 12:51:24.649254 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.649204    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" event={"ID":"f7f370df-beb3-49e8-809a-ffb218bba2c5","Type":"ContainerStarted","Data":"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"}
Sep 19 12:51:24.649407 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.649279    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:24.649407 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.649304    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" event={"ID":"f7f370df-beb3-49e8-809a-ffb218bba2c5","Type":"ContainerStarted","Data":"544cc362cd7468ce41678a9ad8bd3986ab3e7c88c3713f3a4d12f70ccfc4df8c"}
Sep 19 12:51:24.650861 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.650838    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" event={"ID":"0049e67f-330d-45a5-aea1-f77342590e19","Type":"ContainerStarted","Data":"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"}
Sep 19 12:51:24.650977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.650884    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" event={"ID":"0049e67f-330d-45a5-aea1-f77342590e19","Type":"ContainerStarted","Data":"15554fe5b2853af5ff3ec2645b90bacf0ddf6f86ea3eb973a128cdfc5ebf363f"}
Sep 19 12:51:24.650977 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.650904    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:24.666719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.666673    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" podStartSLOduration=1.666659807 podStartE2EDuration="1.666659807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:51:24.665174224 +0000 UTC m=+1568.954729990" watchObservedRunningTime="2026-09-19 12:51:24.666659807 +0000 UTC m=+1568.956215574"
Sep 19 12:51:24.680207 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:24.680157    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" podStartSLOduration=1.6801393 podStartE2EDuration="1.6801393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:51:24.67911656 +0000 UTC m=+1568.968672328" watchObservedRunningTime="2026-09-19 12:51:24.6801393 +0000 UTC m=+1568.969695071"
Sep 19 12:51:29.235252 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.235213    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:51:29.235797 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.235490    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" podUID="0b0a14ea-668c-4473-9409-efd271900763" containerName="payload-processing" containerID="cri-o://8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9" gracePeriod=30
Sep 19 12:51:29.242083 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.242049    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:51:29.242311 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.242287    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" podUID="2047b98b-b833-4c06-9b8d-057de409e66b" containerName="payload-pre-processing" containerID="cri-o://939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b" gracePeriod=30
Sep 19 12:51:29.448594 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.448569    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"]
Sep 19 12:51:29.452466 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.452440    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a2e924-jc8fn"]
Sep 19 12:51:29.489034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.488974    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:51:29.496765 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.496737    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume\") pod \"0b0a14ea-668c-4473-9409-efd271900763\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") "
Sep 19 12:51:29.496894 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.496780    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jxs7\" (UniqueName: \"kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7\") pod \"0b0a14ea-668c-4473-9409-efd271900763\" (UID: \"0b0a14ea-668c-4473-9409-efd271900763\") "
Sep 19 12:51:29.497076 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.497057    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b0a14ea-668c-4473-9409-efd271900763" (UID: "0b0a14ea-668c-4473-9409-efd271900763"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:29.497246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.497231    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0a14ea-668c-4473-9409-efd271900763-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:29.498840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.498815    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7" (OuterVolumeSpecName: "kube-api-access-4jxs7") pod "0b0a14ea-668c-4473-9409-efd271900763" (UID: "0b0a14ea-668c-4473-9409-efd271900763"). InnerVolumeSpecName "kube-api-access-4jxs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:29.516414 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.516372    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:51:29.600366 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.600320    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jxs7\" (UniqueName: \"kubernetes.io/projected/0b0a14ea-668c-4473-9409-efd271900763-kube-api-access-4jxs7\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:29.681637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.681605    2568 generic.go:356] "Generic (PLEG): container finished" podID="2047b98b-b833-4c06-9b8d-057de409e66b" containerID="939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b" exitCode=0
Sep 19 12:51:29.681825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.681649    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"
Sep 19 12:51:29.681825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.681689    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" event={"ID":"2047b98b-b833-4c06-9b8d-057de409e66b","Type":"ContainerDied","Data":"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"}
Sep 19 12:51:29.681825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.681733    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg" event={"ID":"2047b98b-b833-4c06-9b8d-057de409e66b","Type":"ContainerDied","Data":"f7a71d339dfb53773e0754c79dfa59ccce03e0d48a31509ef097e18f5a1eaddb"}
Sep 19 12:51:29.681825 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.681759    2568 scope.go:117] "RemoveContainer" containerID="939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"
Sep 19 12:51:29.683034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.682979    2568 generic.go:356] "Generic (PLEG): container finished" podID="0b0a14ea-668c-4473-9409-efd271900763" containerID="8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9" exitCode=0
Sep 19 12:51:29.683034 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.683032    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"
Sep 19 12:51:29.683177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.683036    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" event={"ID":"0b0a14ea-668c-4473-9409-efd271900763","Type":"ContainerDied","Data":"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"}
Sep 19 12:51:29.683177 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.683076    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk" event={"ID":"0b0a14ea-668c-4473-9409-efd271900763","Type":"ContainerDied","Data":"b86d3d0f453eb843c02bbccaaad2525ce9496dc23bb2b61df9b2fecd90f36305"}
Sep 19 12:51:29.691000 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.690982    2568 scope.go:117] "RemoveContainer" containerID="939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"
Sep 19 12:51:29.691263 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:51:29.691248    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b\": container with ID starting with 939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b not found: ID does not exist" containerID="939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"
Sep 19 12:51:29.691314 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.691271    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b"} err="failed to get container status \"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b\": rpc error: code = NotFound desc = could not find container \"939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b\": container with ID starting with 939db1f3bb4837cd9b272ee2a9475e5f401d57be646cadd4faddef029cf3e74b not found: ID does not exist"
Sep 19 12:51:29.691314 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.691284    2568 scope.go:117] "RemoveContainer" containerID="8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"
Sep 19 12:51:29.698890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.698874    2568 scope.go:117] "RemoveContainer" containerID="8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"
Sep 19 12:51:29.699142 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:51:29.699124    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9\": container with ID starting with 8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9 not found: ID does not exist" containerID="8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"
Sep 19 12:51:29.699187 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.699148    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9"} err="failed to get container status \"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9\": rpc error: code = NotFound desc = could not find container \"8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9\": container with ID starting with 8dc4ff7ea7dbbf80f3abe3c400e765436fec06ed571d567bead647aff5c619f9 not found: ID does not exist"
Sep 19 12:51:29.701523 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.701505    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume\") pod \"2047b98b-b833-4c06-9b8d-057de409e66b\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") "
Sep 19 12:51:29.701603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.701558    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfv58\" (UniqueName: \"kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58\") pod \"2047b98b-b833-4c06-9b8d-057de409e66b\" (UID: \"2047b98b-b833-4c06-9b8d-057de409e66b\") "
Sep 19 12:51:29.701905 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.701882    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2047b98b-b833-4c06-9b8d-057de409e66b" (UID: "2047b98b-b833-4c06-9b8d-057de409e66b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:29.703314 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.703298    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58" (OuterVolumeSpecName: "kube-api-access-vfv58") pod "2047b98b-b833-4c06-9b8d-057de409e66b" (UID: "2047b98b-b833-4c06-9b8d-057de409e66b"). InnerVolumeSpecName "kube-api-access-vfv58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:29.710821 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.710797    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:51:29.713627 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.713608    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-a2e924-6858df79b9-8lsdk"]
Sep 19 12:51:29.722187 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.722167    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0a14ea-668c-4473-9409-efd271900763" path="/var/lib/kubelet/pods/0b0a14ea-668c-4473-9409-efd271900763/volumes"
Sep 19 12:51:29.722540 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.722527    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="895058f3-caaa-49b2-8833-227dc26cec45" path="/var/lib/kubelet/pods/895058f3-caaa-49b2-8833-227dc26cec45/volumes"
Sep 19 12:51:29.802747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.802675    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2047b98b-b833-4c06-9b8d-057de409e66b-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:29.802747 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:29.802695    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfv58\" (UniqueName: \"kubernetes.io/projected/2047b98b-b833-4c06-9b8d-057de409e66b-kube-api-access-vfv58\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:30.007930 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:30.007895    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:51:30.032197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:30.032166    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a2e924-64c4985df4-qlgtg"]
Sep 19 12:51:31.722572 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:31.722523    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2047b98b-b833-4c06-9b8d-057de409e66b" path="/var/lib/kubelet/pods/2047b98b-b833-4c06-9b8d-057de409e66b/volumes"
Sep 19 12:51:31.799464 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:31.799417    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" podUID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" containerName="main" containerID="cri-o://ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37" gracePeriod=45
Sep 19 12:51:31.827142 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:51:31.827113    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd15d36ab_2f40_4a68_9937_9b5c1449e7fe.slice/crio-ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37.scope\": RecentStats: unable to find data in memory cache]"
Sep 19 12:51:32.061105 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.061079    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:51:32.126943 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.126904    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127127 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.126991    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjj7v\" (UniqueName: \"kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127127 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127035    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127127 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127092    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127297 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127177    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127297 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127244    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir\") pod \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\" (UID: \"d15d36ab-2f40-4a68-9937-9b5c1449e7fe\") "
Sep 19 12:51:32.127679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127476    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home" (OuterVolumeSpecName: "home") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.127679 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127672    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-home\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.127878 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127663    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.127878 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.127808    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.130016 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.129993    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:51:32.130016 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.130012    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v" (OuterVolumeSpecName: "kube-api-access-qjj7v") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "kube-api-access-qjj7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:32.130694 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.130656    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm" (OuterVolumeSpecName: "dshm") pod "d15d36ab-2f40-4a68-9937-9b5c1449e7fe" (UID: "d15d36ab-2f40-4a68-9937-9b5c1449e7fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.228475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.228441    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-dshm\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.228475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.228468    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tmp-dir\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.228475 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.228478    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-tls-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.228719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.228487    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjj7v\" (UniqueName: \"kubernetes.io/projected/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-kube-api-access-qjj7v\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.228719 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.228508    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d15d36ab-2f40-4a68-9937-9b5c1449e7fe-model-cache\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.702386 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.702354    2568 generic.go:356] "Generic (PLEG): container finished" podID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" containerID="ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37" exitCode=0
Sep 19 12:51:32.702593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.702397    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"
Sep 19 12:51:32.702593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.702436    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" event={"ID":"d15d36ab-2f40-4a68-9937-9b5c1449e7fe","Type":"ContainerDied","Data":"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"}
Sep 19 12:51:32.702593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.702474    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2" event={"ID":"d15d36ab-2f40-4a68-9937-9b5c1449e7fe","Type":"ContainerDied","Data":"14d9f5facc37a69892af1650aace5227bd4b508c5e39e387dbf0c102b7fdc646"}
Sep 19 12:51:32.702593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.702491    2568 scope.go:117] "RemoveContainer" containerID="ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"
Sep 19 12:51:32.712166 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.712150    2568 scope.go:117] "RemoveContainer" containerID="ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"
Sep 19 12:51:32.712396 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:51:32.712377    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37\": container with ID starting with ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37 not found: ID does not exist" containerID="ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"
Sep 19 12:51:32.712442 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.712406    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37"} err="failed to get container status \"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37\": rpc error: code = NotFound desc = could not find container \"ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37\": container with ID starting with ef93ad3ce5eb0c69a22302658ea4ee6f1b4ea9ecfd06f7f826c8ce2831992b37 not found: ID does not exist"
Sep 19 12:51:32.725336 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.725314    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:51:32.727304 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:32.727284    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/embedding-w0-a2e924-kserve-546f994fbf-2v5p2"]
Sep 19 12:51:33.722196 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:33.722164    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" path="/var/lib/kubelet/pods/d15d36ab-2f40-4a68-9937-9b5c1449e7fe/volumes"
Sep 19 12:51:35.660185 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:35.660158    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:51:35.660624 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:35.660238    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:51:43.500384 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500347    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"]
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500787    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2047b98b-b833-4c06-9b8d-057de409e66b" containerName="payload-pre-processing"
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500799    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2047b98b-b833-4c06-9b8d-057de409e66b" containerName="payload-pre-processing"
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500811    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0a14ea-668c-4473-9409-efd271900763" containerName="payload-processing"
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500817    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0a14ea-668c-4473-9409-efd271900763" containerName="payload-processing"
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500825    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" containerName="main"
Sep 19 12:51:43.500884 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500830    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" containerName="main"
Sep 19 12:51:43.501088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500903    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0a14ea-668c-4473-9409-efd271900763" containerName="payload-processing"
Sep 19 12:51:43.501088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500914    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d15d36ab-2f40-4a68-9937-9b5c1449e7fe" containerName="main"
Sep 19 12:51:43.501088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.500936    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2047b98b-b833-4c06-9b8d-057de409e66b" containerName="payload-pre-processing"
Sep 19 12:51:43.504082 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.504066    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:43.506556 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.506536    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:51:43.507023 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.507010    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:51:43.519754 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.519712    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"]
Sep 19 12:51:43.647309 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.647271    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-49e81a-ml429\" (UID: \"dda77145-b153-4901-b228-6f36eaa91ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:43.748871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.748840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-49e81a-ml429\" (UID: \"dda77145-b153-4901-b228-6f36eaa91ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:43.749418 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.749400    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-49e81a-ml429\" (UID: \"dda77145-b153-4901-b228-6f36eaa91ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:43.814422 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:43.814342    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:44.084696 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:44.084672    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"]
Sep 19 12:51:44.086168 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:51:44.086144    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda77145_b153_4901_b228_6f36eaa91ffc.slice/crio-590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a WatchSource:0}: Error finding container 590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a: Status 404 returned error can't find the container with id 590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a
Sep 19 12:51:44.769737 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:44.769705    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429" event={"ID":"dda77145-b153-4901-b228-6f36eaa91ffc","Type":"ContainerStarted","Data":"9a3c52164ef5eefadb49f52a7de194c00a7a3fdf2e220eda0f6bd066543075ac"}
Sep 19 12:51:44.770067 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:44.769788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429" event={"ID":"dda77145-b153-4901-b228-6f36eaa91ffc","Type":"ContainerStarted","Data":"590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a"}
Sep 19 12:51:45.776375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:45.776342    2568 generic.go:356] "Generic (PLEG): container finished" podID="dda77145-b153-4901-b228-6f36eaa91ffc" containerID="9a3c52164ef5eefadb49f52a7de194c00a7a3fdf2e220eda0f6bd066543075ac" exitCode=0
Sep 19 12:51:45.776813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:45.776426    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429" event={"ID":"dda77145-b153-4901-b228-6f36eaa91ffc","Type":"ContainerDied","Data":"9a3c52164ef5eefadb49f52a7de194c00a7a3fdf2e220eda0f6bd066543075ac"}
Sep 19 12:51:45.913134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:45.913107    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:45.972081 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:45.972055    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca\") pod \"dda77145-b153-4901-b228-6f36eaa91ffc\" (UID: \"dda77145-b153-4901-b228-6f36eaa91ffc\") "
Sep 19 12:51:45.972415 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:45.972387    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dda77145-b153-4901-b228-6f36eaa91ffc" (UID: "dda77145-b153-4901-b228-6f36eaa91ffc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:46.073797 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:46.073727    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda77145-b153-4901-b228-6f36eaa91ffc-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:46.783214 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:46.783183    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"
Sep 19 12:51:46.783600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:46.783231    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429" event={"ID":"dda77145-b153-4901-b228-6f36eaa91ffc","Type":"ContainerDied","Data":"590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a"}
Sep 19 12:51:46.783600 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:46.783274    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="590662b7718ff9a9f02dfdbd54fb9e3af688417b501cd3d7d8fa2a1d2334509a"
Sep 19 12:51:59.537764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.537725    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:51:59.538227 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.537970    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" podUID="ced69de1-1d5f-4038-a8f0-b58376ace694" containerName="payload-pre-processing" containerID="cri-o://0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633" gracePeriod=30
Sep 19 12:51:59.723388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.723362    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"]
Sep 19 12:51:59.723558 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.723397    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-49e81a-ml429"]
Sep 19 12:51:59.797174 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.797111    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:51:59.816620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.816579    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5wp7\" (UniqueName: \"kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7\") pod \"ced69de1-1d5f-4038-a8f0-b58376ace694\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") "
Sep 19 12:51:59.816750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.816674    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume\") pod \"ced69de1-1d5f-4038-a8f0-b58376ace694\" (UID: \"ced69de1-1d5f-4038-a8f0-b58376ace694\") "
Sep 19 12:51:59.817143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.817113    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ced69de1-1d5f-4038-a8f0-b58376ace694" (UID: "ced69de1-1d5f-4038-a8f0-b58376ace694"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:59.819058 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.819027    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7" (OuterVolumeSpecName: "kube-api-access-b5wp7") pod "ced69de1-1d5f-4038-a8f0-b58376ace694" (UID: "ced69de1-1d5f-4038-a8f0-b58376ace694"). InnerVolumeSpecName "kube-api-access-b5wp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:59.854879 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.854847    2568 generic.go:356] "Generic (PLEG): container finished" podID="ced69de1-1d5f-4038-a8f0-b58376ace694" containerID="0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633" exitCode=0
Sep 19 12:51:59.854999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.854891    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"
Sep 19 12:51:59.854999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.854890    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" event={"ID":"ced69de1-1d5f-4038-a8f0-b58376ace694","Type":"ContainerDied","Data":"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"}
Sep 19 12:51:59.854999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.854942    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w" event={"ID":"ced69de1-1d5f-4038-a8f0-b58376ace694","Type":"ContainerDied","Data":"f928d155ad38c98415cbb978ac6bff3532ffbd7ba0964942dbe83ac7c54e0972"}
Sep 19 12:51:59.854999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.854972    2568 scope.go:117] "RemoveContainer" containerID="0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"
Sep 19 12:51:59.866269 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.866248    2568 scope.go:117] "RemoveContainer" containerID="0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"
Sep 19 12:51:59.866661 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:51:59.866636    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633\": container with ID starting with 0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633 not found: ID does not exist" containerID="0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"
Sep 19 12:51:59.866739 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.866670    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633"} err="failed to get container status \"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633\": rpc error: code = NotFound desc = could not find container \"0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633\": container with ID starting with 0286ca66980daf3dbc55ded5d86fa6429ad541ff5abf85618a7be0acc0a90633 not found: ID does not exist"
Sep 19 12:51:59.879734 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.879695    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:51:59.884962 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.884309    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-49e81a-6f68475846-6bw8w"]
Sep 19 12:51:59.921544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.919138    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5wp7\" (UniqueName: \"kubernetes.io/projected/ced69de1-1d5f-4038-a8f0-b58376ace694-kube-api-access-b5wp7\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:51:59.921544 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:51:59.919176    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ced69de1-1d5f-4038-a8f0-b58376ace694-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:01.722644 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:01.722614    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ced69de1-1d5f-4038-a8f0-b58376ace694" path="/var/lib/kubelet/pods/ced69de1-1d5f-4038-a8f0-b58376ace694/volumes"
Sep 19 12:52:01.723017 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:01.722957    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda77145-b153-4901-b228-6f36eaa91ffc" path="/var/lib/kubelet/pods/dda77145-b153-4901-b228-6f36eaa91ffc/volumes"
Sep 19 12:52:06.817379 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:06.817346    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:52:06.817810 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:06.817561    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy" containerID="cri-o://8727baff357f384c09d79324b05861e26deb8bf3d82ddb19c27196f9e6ccc6bb" gracePeriod=30
Sep 19 12:52:07.163860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.163827    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"]
Sep 19 12:52:07.164257 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164245    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ced69de1-1d5f-4038-a8f0-b58376ace694" containerName="payload-pre-processing"
Sep 19 12:52:07.164306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164259    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ced69de1-1d5f-4038-a8f0-b58376ace694" containerName="payload-pre-processing"
Sep 19 12:52:07.164306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164278    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda77145-b153-4901-b228-6f36eaa91ffc" containerName="revoke-keys"
Sep 19 12:52:07.164306 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164284    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda77145-b153-4901-b228-6f36eaa91ffc" containerName="revoke-keys"
Sep 19 12:52:07.164409 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164363    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda77145-b153-4901-b228-6f36eaa91ffc" containerName="revoke-keys"
Sep 19 12:52:07.164409 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.164372    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ced69de1-1d5f-4038-a8f0-b58376ace694" containerName="payload-pre-processing"
Sep 19 12:52:07.168201 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.168185    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:07.170539 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.170519    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:52:07.170603 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.170565    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:52:07.174360 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.174337    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"]
Sep 19 12:52:07.188940 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.188900    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-513a51-hj8fq\" (UID: \"048d082b-d3fb-49f3-adac-f055dd74590c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:07.290197 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.290162    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-513a51-hj8fq\" (UID: \"048d082b-d3fb-49f3-adac-f055dd74590c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:07.290776 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.290757    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-513a51-hj8fq\" (UID: \"048d082b-d3fb-49f3-adac-f055dd74590c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:07.480860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.480763    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:07.760852 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.760813    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"]
Sep 19 12:52:07.763532 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:52:07.763478    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048d082b_d3fb_49f3_adac_f055dd74590c.slice/crio-c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5 WatchSource:0}: Error finding container c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5: Status 404 returned error can't find the container with id c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5
Sep 19 12:52:07.908991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:07.908948    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq" event={"ID":"048d082b-d3fb-49f3-adac-f055dd74590c","Type":"ContainerStarted","Data":"c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5"}
Sep 19 12:52:08.916738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:08.916702    2568 generic.go:356] "Generic (PLEG): container finished" podID="048d082b-d3fb-49f3-adac-f055dd74590c" containerID="652904d4b82febae629916161c245deb1153583b06d7d1991fc183863be55359" exitCode=0
Sep 19 12:52:08.917104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:08.916777    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq" event={"ID":"048d082b-d3fb-49f3-adac-f055dd74590c","Type":"ContainerDied","Data":"652904d4b82febae629916161c245deb1153583b06d7d1991fc183863be55359"}
Sep 19 12:52:10.053837 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.053811    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:10.118802 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.118769    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca\") pod \"048d082b-d3fb-49f3-adac-f055dd74590c\" (UID: \"048d082b-d3fb-49f3-adac-f055dd74590c\") "
Sep 19 12:52:10.119104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.119079    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "048d082b-d3fb-49f3-adac-f055dd74590c" (UID: "048d082b-d3fb-49f3-adac-f055dd74590c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:10.119180 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.119169    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/048d082b-d3fb-49f3-adac-f055dd74590c-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:10.928243 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.928218    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"
Sep 19 12:52:10.928433 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.928261    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq" event={"ID":"048d082b-d3fb-49f3-adac-f055dd74590c","Type":"ContainerDied","Data":"c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5"}
Sep 19 12:52:10.928433 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:10.928315    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1cf651fb61f80686be0f77d1110a434df8e1dc8eff5640b4daacd149e1843b5"
Sep 19 12:52:11.936088 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:11.936052    2568 generic.go:356] "Generic (PLEG): container finished" podID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerID="8727baff357f384c09d79324b05861e26deb8bf3d82ddb19c27196f9e6ccc6bb" exitCode=0
Sep 19 12:52:11.936617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:11.936126    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerDied","Data":"8727baff357f384c09d79324b05861e26deb8bf3d82ddb19c27196f9e6ccc6bb"}
Sep 19 12:52:11.936617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:11.936198    2568 scope.go:117] "RemoveContainer" containerID="d6e6bbcdea8db6007ae5915b5021402c41e41d67150aa7022b2834c348928a3c"
Sep 19 12:52:12.072104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.072079    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:52:12.138942 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.138913    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.138977    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139010    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139091    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139120    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139180    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139223    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139281 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139259    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8ngb\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139299    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket\") pod \"fd095bf6-f086-4215-b50e-e12b8fe81e27\" (UID: \"fd095bf6-f086-4215-b50e-e12b8fe81e27\") "
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139292    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139340    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139537    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139633    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139718    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:12.139762 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.139764    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.141290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.141269    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:52:12.141290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.141267    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:12.141612 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.141584    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb" (OuterVolumeSpecName: "kube-api-access-z8ngb") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "kube-api-access-z8ngb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:12.141612 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.141594    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd095bf6-f086-4215-b50e-e12b8fe81e27" (UID: "fd095bf6-f086-4215-b50e-e12b8fe81e27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:12.241126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241098    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241122    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd095bf6-f086-4215-b50e-e12b8fe81e27-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241126 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241131    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8ngb\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-kube-api-access-z8ngb\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241139    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241148    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241156    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241164    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.241342 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.241172    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd095bf6-f086-4215-b50e-e12b8fe81e27-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:12.942928 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.942894    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5" event={"ID":"fd095bf6-f086-4215-b50e-e12b8fe81e27","Type":"ContainerDied","Data":"b1b1a0889eb8a960ebd3cd6650d5d0d5d88b3b037b80e8d2e138d5029d2acfa0"}
Sep 19 12:52:12.942928 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.942918    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"
Sep 19 12:52:12.943427 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.942950    2568 scope.go:117] "RemoveContainer" containerID="8727baff357f384c09d79324b05861e26deb8bf3d82ddb19c27196f9e6ccc6bb"
Sep 19 12:52:12.966060 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.966038    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:52:12.971948 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:12.971923    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-49e81a-openshift-default-69bddcc5cb-4clr5"]
Sep 19 12:52:13.722809 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:13.722778    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" path="/var/lib/kubelet/pods/fd095bf6-f086-4215-b50e-e12b8fe81e27/volumes"
Sep 19 12:52:18.034395 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:18.034363    2568 scope.go:117] "RemoveContainer" containerID="8506b1e0dbf1ffc534ed5434039bd5a2b82c286eba511e166fa5fe52e2ba8fe7"
Sep 19 12:52:23.128575 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.128540    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:52:23.129134 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.128804    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" podUID="f7f370df-beb3-49e8-809a-ffb218bba2c5" containerName="payload-processing" containerID="cri-o://ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372" gracePeriod=30
Sep 19 12:52:23.141472 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.141445    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:52:23.141857 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.141827    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" podUID="0049e67f-330d-45a5-aea1-f77342590e19" containerName="payload-pre-processing" containerID="cri-o://141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0" gracePeriod=30
Sep 19 12:52:23.299021 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.298880    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"]
Sep 19 12:52:23.302388 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.302081    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-513a51-hj8fq"]
Sep 19 12:52:23.423527 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.423489    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:52:23.426369 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.426343    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:52:23.550506 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.550465    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2vkn\" (UniqueName: \"kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn\") pod \"0049e67f-330d-45a5-aea1-f77342590e19\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") "
Sep 19 12:52:23.550689 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.550605    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume\") pod \"f7f370df-beb3-49e8-809a-ffb218bba2c5\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") "
Sep 19 12:52:23.550732 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.550688    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume\") pod \"0049e67f-330d-45a5-aea1-f77342590e19\" (UID: \"0049e67f-330d-45a5-aea1-f77342590e19\") "
Sep 19 12:52:23.550732 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.550713    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kvnk\" (UniqueName: \"kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk\") pod \"f7f370df-beb3-49e8-809a-ffb218bba2c5\" (UID: \"f7f370df-beb3-49e8-809a-ffb218bba2c5\") "
Sep 19 12:52:23.550954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.550918    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7f370df-beb3-49e8-809a-ffb218bba2c5" (UID: "f7f370df-beb3-49e8-809a-ffb218bba2c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:23.551071 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.551005    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0049e67f-330d-45a5-aea1-f77342590e19" (UID: "0049e67f-330d-45a5-aea1-f77342590e19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:23.552641 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.552611    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn" (OuterVolumeSpecName: "kube-api-access-b2vkn") pod "0049e67f-330d-45a5-aea1-f77342590e19" (UID: "0049e67f-330d-45a5-aea1-f77342590e19"). InnerVolumeSpecName "kube-api-access-b2vkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:23.552870 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.552851    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk" (OuterVolumeSpecName: "kube-api-access-5kvnk") pod "f7f370df-beb3-49e8-809a-ffb218bba2c5" (UID: "f7f370df-beb3-49e8-809a-ffb218bba2c5"). InnerVolumeSpecName "kube-api-access-5kvnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:23.651890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.651808    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f370df-beb3-49e8-809a-ffb218bba2c5-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:23.651890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.651832    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0049e67f-330d-45a5-aea1-f77342590e19-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:23.651890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.651841    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kvnk\" (UniqueName: \"kubernetes.io/projected/f7f370df-beb3-49e8-809a-ffb218bba2c5-kube-api-access-5kvnk\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:23.651890 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.651849    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2vkn\" (UniqueName: \"kubernetes.io/projected/0049e67f-330d-45a5-aea1-f77342590e19-kube-api-access-b2vkn\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:23.722209 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:23.722180    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="048d082b-d3fb-49f3-adac-f055dd74590c" path="/var/lib/kubelet/pods/048d082b-d3fb-49f3-adac-f055dd74590c/volumes"
Sep 19 12:52:24.005430 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.005341    2568 generic.go:356] "Generic (PLEG): container finished" podID="f7f370df-beb3-49e8-809a-ffb218bba2c5" containerID="ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372" exitCode=0
Sep 19 12:52:24.005430 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.005387    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"
Sep 19 12:52:24.005696 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.005427    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" event={"ID":"f7f370df-beb3-49e8-809a-ffb218bba2c5","Type":"ContainerDied","Data":"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"}
Sep 19 12:52:24.005696 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.005475    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8" event={"ID":"f7f370df-beb3-49e8-809a-ffb218bba2c5","Type":"ContainerDied","Data":"544cc362cd7468ce41678a9ad8bd3986ab3e7c88c3713f3a4d12f70ccfc4df8c"}
Sep 19 12:52:24.005696 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.005523    2568 scope.go:117] "RemoveContainer" containerID="ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"
Sep 19 12:52:24.006813 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.006794    2568 generic.go:356] "Generic (PLEG): container finished" podID="0049e67f-330d-45a5-aea1-f77342590e19" containerID="141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0" exitCode=0
Sep 19 12:52:24.006934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.006842    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"
Sep 19 12:52:24.006934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.006880    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" event={"ID":"0049e67f-330d-45a5-aea1-f77342590e19","Type":"ContainerDied","Data":"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"}
Sep 19 12:52:24.007037 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.006936    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8" event={"ID":"0049e67f-330d-45a5-aea1-f77342590e19","Type":"ContainerDied","Data":"15554fe5b2853af5ff3ec2645b90bacf0ddf6f86ea3eb973a128cdfc5ebf363f"}
Sep 19 12:52:24.014303 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.014283    2568 scope.go:117] "RemoveContainer" containerID="ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"
Sep 19 12:52:24.014613 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:52:24.014586    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372\": container with ID starting with ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372 not found: ID does not exist" containerID="ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"
Sep 19 12:52:24.014720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.014624    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372"} err="failed to get container status \"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372\": rpc error: code = NotFound desc = could not find container \"ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372\": container with ID starting with ec8c9e7f4b70252bc43b17a71bb1e613d406a69e4984fab1c1a3affa168ca372 not found: ID does not exist"
Sep 19 12:52:24.014720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.014646    2568 scope.go:117] "RemoveContainer" containerID="141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"
Sep 19 12:52:24.022592 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.022577    2568 scope.go:117] "RemoveContainer" containerID="141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"
Sep 19 12:52:24.022869 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:52:24.022847    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0\": container with ID starting with 141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0 not found: ID does not exist" containerID="141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"
Sep 19 12:52:24.022939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.022881    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0"} err="failed to get container status \"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0\": rpc error: code = NotFound desc = could not find container \"141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0\": container with ID starting with 141a400af22acf737a4583ea1c14652826e3c9599bb23990c741a63660126ea0 not found: ID does not exist"
Sep 19 12:52:24.026596 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.026576    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:52:24.030771 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.030750    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-513a51-58875748fd-b9nh8"]
Sep 19 12:52:24.040242 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.040220    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:52:24.043893 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:24.043871    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-513a51-94c5fbdcc-jh9g8"]
Sep 19 12:52:25.721579 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:25.721545    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0049e67f-330d-45a5-aea1-f77342590e19" path="/var/lib/kubelet/pods/0049e67f-330d-45a5-aea1-f77342590e19/volumes"
Sep 19 12:52:25.721954 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:25.721929    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f370df-beb3-49e8-809a-ffb218bba2c5" path="/var/lib/kubelet/pods/f7f370df-beb3-49e8-809a-ffb218bba2c5/volumes"
Sep 19 12:52:30.161312 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161262    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"]
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161663    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0049e67f-330d-45a5-aea1-f77342590e19" containerName="payload-pre-processing"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161675    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0049e67f-330d-45a5-aea1-f77342590e19" containerName="payload-pre-processing"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161690    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161696    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161703    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048d082b-d3fb-49f3-adac-f055dd74590c" containerName="revoke-keys"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161708    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="048d082b-d3fb-49f3-adac-f055dd74590c" containerName="revoke-keys"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161718    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161723    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161734    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f370df-beb3-49e8-809a-ffb218bba2c5" containerName="payload-processing"
Sep 19 12:52:30.161738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161739    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f370df-beb3-49e8-809a-ffb218bba2c5" containerName="payload-processing"
Sep 19 12:52:30.162085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161795    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:30.162085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161803    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f370df-beb3-49e8-809a-ffb218bba2c5" containerName="payload-processing"
Sep 19 12:52:30.162085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161812    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="048d082b-d3fb-49f3-adac-f055dd74590c" containerName="revoke-keys"
Sep 19 12:52:30.162085 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.161819    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0049e67f-330d-45a5-aea1-f77342590e19" containerName="payload-pre-processing"
Sep 19 12:52:30.166237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.166219    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:30.168883 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.168860    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:52:30.169022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.168869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:52:30.172964 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.172941    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"]
Sep 19 12:52:30.318317 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.318278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4\" (UID: \"1b681be0-f6c7-4460-ba14-cec55024838b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:30.419258 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.419173    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4\" (UID: \"1b681be0-f6c7-4460-ba14-cec55024838b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:30.419854 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.419833    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4\" (UID: \"1b681be0-f6c7-4460-ba14-cec55024838b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:30.479779 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.479735    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:30.750803 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:30.750779    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"]
Sep 19 12:52:30.752997 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:52:30.752973    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b681be0_f6c7_4460_ba14_cec55024838b.slice/crio-4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1 WatchSource:0}: Error finding container 4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1: Status 404 returned error can't find the container with id 4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1
Sep 19 12:52:31.048806 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:31.048717    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4" event={"ID":"1b681be0-f6c7-4460-ba14-cec55024838b","Type":"ContainerStarted","Data":"4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1"}
Sep 19 12:52:32.055551 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:32.055520    2568 generic.go:356] "Generic (PLEG): container finished" podID="1b681be0-f6c7-4460-ba14-cec55024838b" containerID="0e99bed13cd3d63f184aa0e48d43237b1adcf7fb83b92b902f3db3ceae636dcd" exitCode=0
Sep 19 12:52:32.055973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:32.055591    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4" event={"ID":"1b681be0-f6c7-4460-ba14-cec55024838b","Type":"ContainerDied","Data":"0e99bed13cd3d63f184aa0e48d43237b1adcf7fb83b92b902f3db3ceae636dcd"}
Sep 19 12:52:33.214451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:33.214424    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:33.347995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:33.347910    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca\") pod \"1b681be0-f6c7-4460-ba14-cec55024838b\" (UID: \"1b681be0-f6c7-4460-ba14-cec55024838b\") "
Sep 19 12:52:33.348249 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:33.348227    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b681be0-f6c7-4460-ba14-cec55024838b" (UID: "1b681be0-f6c7-4460-ba14-cec55024838b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:33.448991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:33.448955    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b681be0-f6c7-4460-ba14-cec55024838b-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:34.067833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:34.067735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4" event={"ID":"1b681be0-f6c7-4460-ba14-cec55024838b","Type":"ContainerDied","Data":"4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1"}
Sep 19 12:52:34.067833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:34.067755    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"
Sep 19 12:52:34.067833 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:34.067791    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ff5b64beb8782cbef1fb1d1d0dcf390f0452a70c0acdd37512aa4b2d9537ec1"
Sep 19 12:52:40.943375 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:40.943344    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:52:40.944013 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:40.943567    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" podUID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" containerName="payload-pre-processing" containerID="cri-o://f7783f19f117b5752fbbb3f4adda8fcdf1c3c5471a2d1fad37bfb5c04217eafc" gracePeriod=30
Sep 19 12:52:41.108675 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.108630    2568 generic.go:356] "Generic (PLEG): container finished" podID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" containerID="f7783f19f117b5752fbbb3f4adda8fcdf1c3c5471a2d1fad37bfb5c04217eafc" exitCode=0
Sep 19 12:52:41.108675 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.108673    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" event={"ID":"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e","Type":"ContainerDied","Data":"f7783f19f117b5752fbbb3f4adda8fcdf1c3c5471a2d1fad37bfb5c04217eafc"}
Sep 19 12:52:41.123794 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.123748    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"]
Sep 19 12:52:41.127006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.126982    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2a487c-z6bz4"]
Sep 19 12:52:41.208937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.208862    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:52:41.328093 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.328049    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume\") pod \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") "
Sep 19 12:52:41.328263 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.328142    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvlw5\" (UniqueName: \"kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5\") pod \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\" (UID: \"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e\") "
Sep 19 12:52:41.328410 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.328386    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" (UID: "d0e40fae-5cd8-4d93-b53e-0b378b2eee1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:41.328555 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.328541    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:41.330264 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.330223    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5" (OuterVolumeSpecName: "kube-api-access-xvlw5") pod "d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" (UID: "d0e40fae-5cd8-4d93-b53e-0b378b2eee1e"). InnerVolumeSpecName "kube-api-access-xvlw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:41.430006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.429964    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvlw5\" (UniqueName: \"kubernetes.io/projected/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e-kube-api-access-xvlw5\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:41.722394 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:41.722359    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b681be0-f6c7-4460-ba14-cec55024838b" path="/var/lib/kubelet/pods/1b681be0-f6c7-4460-ba14-cec55024838b/volumes"
Sep 19 12:52:42.115692 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:42.115606    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"
Sep 19 12:52:42.115692 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:42.115620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms" event={"ID":"d0e40fae-5cd8-4d93-b53e-0b378b2eee1e","Type":"ContainerDied","Data":"6734335c5ca9fe6a22cbab0a269f40508bbd4612ebbd7d6eb2c6d958b84d9df9"}
Sep 19 12:52:42.116135 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:42.115708    2568 scope.go:117] "RemoveContainer" containerID="f7783f19f117b5752fbbb3f4adda8fcdf1c3c5471a2d1fad37bfb5c04217eafc"
Sep 19 12:52:42.132855 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:42.132833    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:52:42.134906 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:42.134887    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2a487c-56769d894b-spzms"]
Sep 19 12:52:43.722732 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:43.722698    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" path="/var/lib/kubelet/pods/d0e40fae-5cd8-4d93-b53e-0b378b2eee1e/volumes"
Sep 19 12:52:47.928331 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:47.928292    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:52:47.928774 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:47.928601    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy" containerID="cri-o://33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e" gracePeriod=30
Sep 19 12:52:48.256672 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.256581    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"]
Sep 19 12:52:48.257001 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.256989    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" containerName="payload-pre-processing"
Sep 19 12:52:48.257045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257002    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" containerName="payload-pre-processing"
Sep 19 12:52:48.257045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257016    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b681be0-f6c7-4460-ba14-cec55024838b" containerName="revoke-keys"
Sep 19 12:52:48.257045 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257022    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b681be0-f6c7-4460-ba14-cec55024838b" containerName="revoke-keys"
Sep 19 12:52:48.257141 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257082    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e40fae-5cd8-4d93-b53e-0b378b2eee1e" containerName="payload-pre-processing"
Sep 19 12:52:48.257141 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257090    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b681be0-f6c7-4460-ba14-cec55024838b" containerName="revoke-keys"
Sep 19 12:52:48.257141 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.257101    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd095bf6-f086-4215-b50e-e12b8fe81e27" containerName="istio-proxy"
Sep 19 12:52:48.261812 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.261796    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:48.265027 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.265001    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:52:48.265181 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.265092    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:52:48.269740 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.269712    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"]
Sep 19 12:52:48.297925 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.297898    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn\" (UID: \"4c77683b-51f5-4d4f-bdf1-34d891260aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:48.399417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.399385    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn\" (UID: \"4c77683b-51f5-4d4f-bdf1-34d891260aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:48.400007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.399987    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn\" (UID: \"4c77683b-51f5-4d4f-bdf1-34d891260aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:48.581486 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.581394    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:48.852450 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:48.852423    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"]
Sep 19 12:52:48.855002 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:52:48.854974    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c77683b_51f5_4d4f_bdf1_34d891260aa0.slice/crio-fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934 WatchSource:0}: Error finding container fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934: Status 404 returned error can't find the container with id fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934
Sep 19 12:52:49.154283 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:49.154251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn" event={"ID":"4c77683b-51f5-4d4f-bdf1-34d891260aa0","Type":"ContainerStarted","Data":"fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934"}
Sep 19 12:52:50.161981 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:50.161946    2568 generic.go:356] "Generic (PLEG): container finished" podID="4c77683b-51f5-4d4f-bdf1-34d891260aa0" containerID="6831bf4829cfaac5dc5ad3c23a88990db9a51c93a10cc188a377e12b8b9e7068" exitCode=0
Sep 19 12:52:50.162389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:50.162036    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn" event={"ID":"4c77683b-51f5-4d4f-bdf1-34d891260aa0","Type":"ContainerDied","Data":"6831bf4829cfaac5dc5ad3c23a88990db9a51c93a10cc188a377e12b8b9e7068"}
Sep 19 12:52:51.304101 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:51.304075    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:51.327358 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:51.327329    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca\") pod \"4c77683b-51f5-4d4f-bdf1-34d891260aa0\" (UID: \"4c77683b-51f5-4d4f-bdf1-34d891260aa0\") "
Sep 19 12:52:51.327775 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:51.327739    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c77683b-51f5-4d4f-bdf1-34d891260aa0" (UID: "4c77683b-51f5-4d4f-bdf1-34d891260aa0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:51.428894 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:51.428809    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c77683b-51f5-4d4f-bdf1-34d891260aa0-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:52.174292 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:52.174260    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"
Sep 19 12:52:52.174445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:52.174287    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn" event={"ID":"4c77683b-51f5-4d4f-bdf1-34d891260aa0","Type":"ContainerDied","Data":"fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934"}
Sep 19 12:52:52.174445 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:52.174336    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc6f53c9811f5fd576291ac00b09bf228c038f5b34aa5fc16ad1af5ce3807934"
Sep 19 12:52:53.178321 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.178293    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:52:53.180514 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.180475    2568 generic.go:356] "Generic (PLEG): container finished" podID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerID="33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e" exitCode=0
Sep 19 12:52:53.180651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.180531    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"
Sep 19 12:52:53.180651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.180565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerDied","Data":"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"}
Sep 19 12:52:53.180651 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.180629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96" event={"ID":"308d75b3-ebd4-45c3-96b2-047a11e12b6c","Type":"ContainerDied","Data":"3b78b1c735e92294b7b0c66753df934b8ed09f64b419c7495daa596b1d07a2db"}
Sep 19 12:52:53.180814 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.180661    2568 scope.go:117] "RemoveContainer" containerID="33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"
Sep 19 12:52:53.189298 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.189254    2568 scope.go:117] "RemoveContainer" containerID="b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"
Sep 19 12:52:53.197320 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.197305    2568 scope.go:117] "RemoveContainer" containerID="33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"
Sep 19 12:52:53.197635 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:52:53.197606    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e\": container with ID starting with 33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e not found: ID does not exist" containerID="33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"
Sep 19 12:52:53.197764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.197643    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e"} err="failed to get container status \"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e\": rpc error: code = NotFound desc = could not find container \"33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e\": container with ID starting with 33033ba374c0cff900ad45bdc31892cb1545068dcb4f0909432b228fe9ee3b3e not found: ID does not exist"
Sep 19 12:52:53.197764 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.197662    2568 scope.go:117] "RemoveContainer" containerID="b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"
Sep 19 12:52:53.198866 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:52:53.198818    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48\": container with ID starting with b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48 not found: ID does not exist" containerID="b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"
Sep 19 12:52:53.199012 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.198877    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48"} err="failed to get container status \"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48\": rpc error: code = NotFound desc = could not find container \"b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48\": container with ID starting with b799cf15fd3890b3207a99ff497fd298d05fdddd25e8cfaa0e2aed28318eeb48 not found: ID does not exist"
Sep 19 12:52:53.247911 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.247871    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.247926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.247961    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248033    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k7nx\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248074    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248115 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248113    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248141    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248173    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:53.248374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248204    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248374 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248261    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket\") pod \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\" (UID: \"308d75b3-ebd4-45c3-96b2-047a11e12b6c\") "
Sep 19 12:52:53.248615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248391    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:53.248615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248538    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:53.248724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248627    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:53.248724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248654    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.248724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248674    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.248724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248675    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:53.248724 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.248688    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.250278 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.250252    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:53.250381 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.250343    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:53.250612 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.250590    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx" (OuterVolumeSpecName: "kube-api-access-9k7nx") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "kube-api-access-9k7nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:53.250756 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.250734    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "308d75b3-ebd4-45c3-96b2-047a11e12b6c" (UID: "308d75b3-ebd4-45c3-96b2-047a11e12b6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:52:53.350121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350084    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.350121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350113    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.350121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350123    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.350121 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350132    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.350367 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350141    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d75b3-ebd4-45c3-96b2-047a11e12b6c-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.350367 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.350149    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k7nx\" (UniqueName: \"kubernetes.io/projected/308d75b3-ebd4-45c3-96b2-047a11e12b6c-kube-api-access-9k7nx\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:52:53.503939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.503909    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:52:53.507301 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.507276    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a487c-openshift-default-565b9b7f5b-r6f96"]
Sep 19 12:52:53.722385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:53.722354    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" path="/var/lib/kubelet/pods/308d75b3-ebd4-45c3-96b2-047a11e12b6c/volumes"
Sep 19 12:52:58.958683 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:58.958625    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"]
Sep 19 12:52:58.960684 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:58.960661    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5068d9-29830365-zbpwp"]
Sep 19 12:52:59.183371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:59.183338    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"]
Sep 19 12:52:59.186270 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:59.186245    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5068d9-6dwfn"]
Sep 19 12:52:59.726912 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:59.726863    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c77683b-51f5-4d4f-bdf1-34d891260aa0" path="/var/lib/kubelet/pods/4c77683b-51f5-4d4f-bdf1-34d891260aa0/volumes"
Sep 19 12:52:59.727349 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:52:59.727330    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e" path="/var/lib/kubelet/pods/a6e9260f-d2fd-4e9b-8a73-eccfcb16d57e/volumes"
Sep 19 12:53:05.661360 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:05.661330    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:53:05.661820 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:05.661544    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy" containerID="cri-o://291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748" gracePeriod=30
Sep 19 12:53:10.906360 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:10.906331    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:53:11.019744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019665    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.019744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019715    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.019744 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019744    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j78mn\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019780    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019804    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019832    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019874    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019938    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019970    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket\") pod \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\" (UID: \"1ea661d4-d732-4162-b95a-a94e0ff79ac9\") "
Sep 19 12:53:11.020022 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.019950    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:11.020385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020038    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:11.020385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020231    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:11.020385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020333    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:11.020385 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020385    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.020637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020401    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.020637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020413    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.020637 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.020514    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:53:11.022048 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.022022    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:53:11.022162 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.022102    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:11.022428 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.022406    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:11.022469 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.022439    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn" (OuterVolumeSpecName: "kube-api-access-j78mn") pod "1ea661d4-d732-4162-b95a-a94e0ff79ac9" (UID: "1ea661d4-d732-4162-b95a-a94e0ff79ac9"). InnerVolumeSpecName "kube-api-access-j78mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:11.121348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121315    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.121348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121341    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.121348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121350    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j78mn\" (UniqueName: \"kubernetes.io/projected/1ea661d4-d732-4162-b95a-a94e0ff79ac9-kube-api-access-j78mn\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.121673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121359    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.121673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121368    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.121673 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.121376    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea661d4-d732-4162-b95a-a94e0ff79ac9-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:11.286399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.286310    2568 generic.go:356] "Generic (PLEG): container finished" podID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerID="291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748" exitCode=0
Sep 19 12:53:11.286399 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.286364    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"
Sep 19 12:53:11.286615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.286398    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerDied","Data":"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"}
Sep 19 12:53:11.286615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.286454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2" event={"ID":"1ea661d4-d732-4162-b95a-a94e0ff79ac9","Type":"ContainerDied","Data":"c6d68846bbdd8b30c718fc8d7da8c14bb59d3f3c5c64525bc306d1fa7b6c3808"}
Sep 19 12:53:11.286615 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.286478    2568 scope.go:117] "RemoveContainer" containerID="291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"
Sep 19 12:53:11.296174 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.296153    2568 scope.go:117] "RemoveContainer" containerID="46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"
Sep 19 12:53:11.305364 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.305350    2568 scope.go:117] "RemoveContainer" containerID="291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"
Sep 19 12:53:11.305632 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:53:11.305616    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748\": container with ID starting with 291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748 not found: ID does not exist" containerID="291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"
Sep 19 12:53:11.305692 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.305640    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748"} err="failed to get container status \"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748\": rpc error: code = NotFound desc = could not find container \"291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748\": container with ID starting with 291877fc7990deb7f1b6fb25d8e60e7e8e183074ff2affa64753661a628fd748 not found: ID does not exist"
Sep 19 12:53:11.305692 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.305657    2568 scope.go:117] "RemoveContainer" containerID="46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"
Sep 19 12:53:11.305903 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:53:11.305887    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91\": container with ID starting with 46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91 not found: ID does not exist" containerID="46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"
Sep 19 12:53:11.305945 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.305906    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91"} err="failed to get container status \"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91\": rpc error: code = NotFound desc = could not find container \"46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91\": container with ID starting with 46c589432fe38c9f6204b4556b06acc71d06029d89f71995fb770070421c3b91 not found: ID does not exist"
Sep 19 12:53:11.310363 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.310340    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:53:11.316309 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.316288    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5068d9-openshift-default-7789b4b48-mtdx2"]
Sep 19 12:53:11.722237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:11.722201    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" path="/var/lib/kubelet/pods/1ea661d4-d732-4162-b95a-a94e0ff79ac9/volumes"
Sep 19 12:53:12.674940 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:12.674906    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:53:18.072484 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:18.072453    2568 scope.go:117] "RemoveContainer" containerID="c7896147e768116417675d92e3631a5a525d995e4160f23bf42b1701524a7737"
Sep 19 12:53:18.080875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:18.080849    2568 scope.go:117] "RemoveContainer" containerID="bce581ab8b01e9ca33f7a8a4dbe53cbda72534f70d15a2edd38520811035caa3"
Sep 19 12:53:24.492904 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.492867    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"]
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493226    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493237    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493247    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493254    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493269    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493274    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493288    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c77683b-51f5-4d4f-bdf1-34d891260aa0" containerName="revoke-keys"
Sep 19 12:53:24.493326 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493293    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c77683b-51f5-4d4f-bdf1-34d891260aa0" containerName="revoke-keys"
Sep 19 12:53:24.493620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493346    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c77683b-51f5-4d4f-bdf1-34d891260aa0" containerName="revoke-keys"
Sep 19 12:53:24.493620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493354    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:53:24.493620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493362    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.493620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493369    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:53:24.493620 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.493375    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="308d75b3-ebd4-45c3-96b2-047a11e12b6c" containerName="istio-proxy"
Sep 19 12:53:24.498113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.498096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:24.500488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.500466    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:53:24.500636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.500468    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:53:24.504354 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.504331    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"]
Sep 19 12:53:24.554280 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.554246    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb\" (UID: \"497227e8-f7ef-4daa-bb9e-9d294cfb7a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:24.655104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.655072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb\" (UID: \"497227e8-f7ef-4daa-bb9e-9d294cfb7a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:24.655712 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.655691    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb\" (UID: \"497227e8-f7ef-4daa-bb9e-9d294cfb7a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:24.810113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:24.810037    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:25.081322 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:25.081294    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"]
Sep 19 12:53:25.083311 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:53:25.083284    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497227e8_f7ef_4daa_bb9e_9d294cfb7a97.slice/crio-d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024 WatchSource:0}: Error finding container d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024: Status 404 returned error can't find the container with id d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024
Sep 19 12:53:25.364646 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:25.364614    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb" event={"ID":"497227e8-f7ef-4daa-bb9e-9d294cfb7a97","Type":"ContainerStarted","Data":"d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024"}
Sep 19 12:53:26.370545 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:26.370505    2568 generic.go:356] "Generic (PLEG): container finished" podID="497227e8-f7ef-4daa-bb9e-9d294cfb7a97" containerID="f234797ab386a9b2abd536828953cf2bef4c71cae87796d49da2f12dbbbdf13e" exitCode=0
Sep 19 12:53:26.371068 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:26.370557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb" event={"ID":"497227e8-f7ef-4daa-bb9e-9d294cfb7a97","Type":"ContainerDied","Data":"f234797ab386a9b2abd536828953cf2bef4c71cae87796d49da2f12dbbbdf13e"}
Sep 19 12:53:27.528367 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.528338    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:27.587999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.587967    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca\") pod \"497227e8-f7ef-4daa-bb9e-9d294cfb7a97\" (UID: \"497227e8-f7ef-4daa-bb9e-9d294cfb7a97\") "
Sep 19 12:53:27.588309 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.588287    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "497227e8-f7ef-4daa-bb9e-9d294cfb7a97" (UID: "497227e8-f7ef-4daa-bb9e-9d294cfb7a97"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:53:27.689294 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.689209    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/497227e8-f7ef-4daa-bb9e-9d294cfb7a97-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:27.693450 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.693418    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" podUID="97162928-fdf3-4c9f-9321-846e87911a3d" containerName="main" containerID="cri-o://db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b" gracePeriod=45
Sep 19 12:53:27.932720 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.932696    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:53:27.991939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.991842    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkf7v\" (UniqueName: \"kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.991939 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.991886    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.992163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.991991    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.992163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992021    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.992163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992039    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.992163 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992059    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache\") pod \"97162928-fdf3-4c9f-9321-846e87911a3d\" (UID: \"97162928-fdf3-4c9f-9321-846e87911a3d\") "
Sep 19 12:53:27.992383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992272    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home" (OuterVolumeSpecName: "home") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:27.992383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992295    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:27.992383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.992346    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache" (OuterVolumeSpecName: "model-cache") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:27.994104 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.994080    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:27.994211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.994113    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm" (OuterVolumeSpecName: "dshm") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:27.994211 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:27.994115    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v" (OuterVolumeSpecName: "kube-api-access-zkf7v") pod "97162928-fdf3-4c9f-9321-846e87911a3d" (UID: "97162928-fdf3-4c9f-9321-846e87911a3d"). InnerVolumeSpecName "kube-api-access-zkf7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:28.093723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093694    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-model-cache\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.093723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093720    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkf7v\" (UniqueName: \"kubernetes.io/projected/97162928-fdf3-4c9f-9321-846e87911a3d-kube-api-access-zkf7v\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.093927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093731    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97162928-fdf3-4c9f-9321-846e87911a3d-tls-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.093927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093743    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-dshm\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.093927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093751    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-home\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.093927 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.093759    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97162928-fdf3-4c9f-9321-846e87911a3d-tmp-dir\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:53:28.382305 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.382267    2568 generic.go:356] "Generic (PLEG): container finished" podID="97162928-fdf3-4c9f-9321-846e87911a3d" containerID="db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b" exitCode=0
Sep 19 12:53:28.382520 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.382310    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"
Sep 19 12:53:28.382520 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.382311    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" event={"ID":"97162928-fdf3-4c9f-9321-846e87911a3d","Type":"ContainerDied","Data":"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"}
Sep 19 12:53:28.382520 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.382379    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd" event={"ID":"97162928-fdf3-4c9f-9321-846e87911a3d","Type":"ContainerDied","Data":"8ee9b08faac2fd8bdf76b2d90272380ff32f15f94d448aa21fb3f42c2c862f9f"}
Sep 19 12:53:28.382520 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.382404    2568 scope.go:117] "RemoveContainer" containerID="db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"
Sep 19 12:53:28.383960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.383940    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"
Sep 19 12:53:28.384025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.383961    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb" event={"ID":"497227e8-f7ef-4daa-bb9e-9d294cfb7a97","Type":"ContainerDied","Data":"d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024"}
Sep 19 12:53:28.384025 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.384019    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d80d9ea37f32a86765f458d50549f7119d9b4aa909d3e8a1cbcb74f843e1f024"
Sep 19 12:53:28.391670 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.391654    2568 scope.go:117] "RemoveContainer" containerID="db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"
Sep 19 12:53:28.391945 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:53:28.391924    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b\": container with ID starting with db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b not found: ID does not exist" containerID="db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"
Sep 19 12:53:28.391991 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.391952    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b"} err="failed to get container status \"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b\": rpc error: code = NotFound desc = could not find container \"db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b\": container with ID starting with db6763a76a77ff53081ede8a220670b2d20ac364b15959935c260d200fec669b not found: ID does not exist"
Sep 19 12:53:28.407759 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.407736    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:53:28.410811 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:28.410790    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/auth-test-model-b0e30e-kserve-f4797cbc-ws7xd"]
Sep 19 12:53:29.721353 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:29.721317    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97162928-fdf3-4c9f-9321-846e87911a3d" path="/var/lib/kubelet/pods/97162928-fdf3-4c9f-9321-846e87911a3d/volumes"
Sep 19 12:53:40.160840 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:40.160801    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"]
Sep 19 12:53:40.166629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:40.166601    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-04a668-29830365-8j4kn"]
Sep 19 12:53:40.414143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:40.414048    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"]
Sep 19 12:53:40.422999 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:40.422962    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-04a668-ckbqb"]
Sep 19 12:53:41.722052 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:41.722016    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="497227e8-f7ef-4daa-bb9e-9d294cfb7a97" path="/var/lib/kubelet/pods/497227e8-f7ef-4daa-bb9e-9d294cfb7a97/volumes"
Sep 19 12:53:41.722439 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:53:41.722298    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53069d6c-22da-4289-b9b4-39f6e2568d7c" path="/var/lib/kubelet/pods/53069d6c-22da-4289-b9b4-39f6e2568d7c/volumes"
Sep 19 12:54:18.122782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:54:18.122752    2568 scope.go:117] "RemoveContainer" containerID="979bd092bc0a94e75703d8384fd2bb8562f1b2dcfb2dc2d0456a5fa29219f036"
Sep 19 12:54:18.131372 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:54:18.131353    2568 scope.go:117] "RemoveContainer" containerID="8fd060990abcb75e541411e48e2952acf47b50948b337311716f6bf2e485513c"
Sep 19 12:55:17.771235 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:55:17.771209    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:55:17.774560 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:55:17.774537    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 12:56:17.880253 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880223    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880609    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880621    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea661d4-d732-4162-b95a-a94e0ff79ac9" containerName="istio-proxy"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880632    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="497227e8-f7ef-4daa-bb9e-9d294cfb7a97" containerName="revoke-keys"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880637    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="497227e8-f7ef-4daa-bb9e-9d294cfb7a97" containerName="revoke-keys"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880646    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97162928-fdf3-4c9f-9321-846e87911a3d" containerName="main"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880652    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="97162928-fdf3-4c9f-9321-846e87911a3d" containerName="main"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880703    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="497227e8-f7ef-4daa-bb9e-9d294cfb7a97" containerName="revoke-keys"
Sep 19 12:56:17.882417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.880712    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="97162928-fdf3-4c9f-9321-846e87911a3d" containerName="main"
Sep 19 12:56:17.883402 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.883388    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:17.885723 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.885705    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f32129-dockercfg-59jpq\""
Sep 19 12:56:17.886078 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.886063    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f32129\""
Sep 19 12:56:17.900397 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.900373    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:56:17.982858 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.982821    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:17.983059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:17.982901    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfqmb\" (UniqueName: \"kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.083909 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.083868    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.084086 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.083975    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfqmb\" (UniqueName: \"kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.084629 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.084610    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.092421 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.092398    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfqmb\" (UniqueName: \"kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb\") pod \"payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.193783 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.193697    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:18.471937 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:18.471910    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:56:18.473993 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:56:18.473963    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f0555d_3ee9_4606_8b5c_03507cd9dc26.slice/crio-fea0794f12a782cf26f702431faba3059e607034e21548c36df8a988680794bd WatchSource:0}: Error finding container fea0794f12a782cf26f702431faba3059e607034e21548c36df8a988680794bd: Status 404 returned error can't find the container with id fea0794f12a782cf26f702431faba3059e607034e21548c36df8a988680794bd
Sep 19 12:56:19.235424 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:19.235381    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" event={"ID":"35f0555d-3ee9-4606-8b5c-03507cd9dc26","Type":"ContainerStarted","Data":"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"}
Sep 19 12:56:19.236143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:19.235474    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:19.236143 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:19.235514    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" event={"ID":"35f0555d-3ee9-4606-8b5c-03507cd9dc26","Type":"ContainerStarted","Data":"fea0794f12a782cf26f702431faba3059e607034e21548c36df8a988680794bd"}
Sep 19 12:56:19.253008 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:19.252960    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" podStartSLOduration=2.252945945 podStartE2EDuration="2.252945945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:19.250441635 +0000 UTC m=+1863.539997404" watchObservedRunningTime="2026-09-19 12:56:19.252945945 +0000 UTC m=+1863.542501714"
Sep 19 12:56:30.243440 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:30.243358    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:56:52.358661 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.358630    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 12:56:52.362216 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.362199    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.365006 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.364984    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-4e754a\""
Sep 19 12:56:52.365118 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.365011    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-4e754a-dockercfg-vgdc7\""
Sep 19 12:56:52.378686 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.378661    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 12:56:52.408562 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.408532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlmf9\" (UniqueName: \"kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.408715 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.408579    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.509639 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.509603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlmf9\" (UniqueName: \"kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.509639 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.509642    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.510358 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.510336    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.517957 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.517925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlmf9\" (UniqueName: \"kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9\") pod \"payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.672284 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.672240    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:52.944389 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:52.944365    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 12:56:52.947033 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:56:52.947008    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e04786_492c_4fc7_a065_0e8e27c1648e.slice/crio-077124c3b4f714a47c19b9b1b2457cc4b0a3a8835bade9fd396a59ddf517fe7e WatchSource:0}: Error finding container 077124c3b4f714a47c19b9b1b2457cc4b0a3a8835bade9fd396a59ddf517fe7e: Status 404 returned error can't find the container with id 077124c3b4f714a47c19b9b1b2457cc4b0a3a8835bade9fd396a59ddf517fe7e
Sep 19 12:56:53.405448 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:53.405409    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" event={"ID":"56e04786-492c-4fc7-a065-0e8e27c1648e","Type":"ContainerStarted","Data":"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"}
Sep 19 12:56:53.405934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:53.405467    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" event={"ID":"56e04786-492c-4fc7-a065-0e8e27c1648e","Type":"ContainerStarted","Data":"077124c3b4f714a47c19b9b1b2457cc4b0a3a8835bade9fd396a59ddf517fe7e"}
Sep 19 12:56:53.405934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:53.405512    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:56:53.446246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:56:53.446195    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" podStartSLOduration=1.4461796439999999 podStartE2EDuration="1.446179644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:53.445749903 +0000 UTC m=+1897.735305674" watchObservedRunningTime="2026-09-19 12:56:53.446179644 +0000 UTC m=+1897.735735412"
Sep 19 12:57:04.413167 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:57:04.413136    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 12:57:18.172246 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:57:18.172213    2568 scope.go:117] "RemoveContainer" containerID="0491eb198c641a18447867b33021157820633098b93e8374dda3e231f80bb090"
Sep 19 12:57:24.840070 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:57:24.840038    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 12:58:01.052852 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.052749    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 12:58:01.820237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.820207    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:01.823329 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.823305    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.825871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.825846    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-1593f0-openshift-default-dockercfg-5gw8d\""
Sep 19 12:58:01.835290 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.835270    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:01.859448 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859419    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kl7b\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859469    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859523    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859617 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859555    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859684    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859782 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859778    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859809    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.859896 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.859825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961114 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961077    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961178    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961276    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961310 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961312    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl7b\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961699 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.961903 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.961830    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.962027 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.962006    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.964050 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.964029    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.964251 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.964233    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.969157 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.969135    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:01.969371 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:01.969352    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl7b\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b\") pod \"e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") " pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:02.134536 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.134483    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:02.413149 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.413118    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:02.415191 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:58:02.415166    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf048b2_93cf_4c8d_a12e_d8280881c434.slice/crio-dd2f0663732ae8c81b23d61ed72b7c054d732424823e06872a7aa7205299b278 WatchSource:0}: Error finding container dd2f0663732ae8c81b23d61ed72b7c054d732424823e06872a7aa7205299b278: Status 404 returned error can't find the container with id dd2f0663732ae8c81b23d61ed72b7c054d732424823e06872a7aa7205299b278
Sep 19 12:58:02.417674 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.417637    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:02.417770 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.417726    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:02.417810 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.417766    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:02.751871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.751781    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerStarted","Data":"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"}
Sep 19 12:58:02.751871 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:02.751831    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerStarted","Data":"dd2f0663732ae8c81b23d61ed72b7c054d732424823e06872a7aa7205299b278"}
Sep 19 12:58:03.135636 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:03.135600    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:03.141125 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:03.141102    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:03.160832 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:03.160786    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podStartSLOduration=2.160771018 podStartE2EDuration="2.160771018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:58:02.796152252 +0000 UTC m=+1967.085708021" watchObservedRunningTime="2026-09-19 12:58:03.160771018 +0000 UTC m=+1967.450326778"
Sep 19 12:58:03.757001 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:03.756969    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:03.757899 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:03.757880    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:08.591553 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.591480    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:08.596461 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.596438    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.598973 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.598951    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-1593f0-dockercfg-bl7nb\""
Sep 19 12:58:08.599106 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.598971    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-1593f0\""
Sep 19 12:58:08.603785 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.603765    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:08.626059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.626022    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dm7\" (UniqueName: \"kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.626191 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.626133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.727123 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.727089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dm7\" (UniqueName: \"kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.727286 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.727142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.727841 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.727821    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.735301 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.735274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dm7\" (UniqueName: \"kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7\") pod \"payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:08.908080 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:08.908049    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:09.178323 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:09.178300    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:09.179885 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:58:09.179860    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94654102_6b7c_4781_894a_2973995e77b4.slice/crio-bbd2cdda1e07206e3957303f7ae299be054b032e30d8f5690f7124c89087739a WatchSource:0}: Error finding container bbd2cdda1e07206e3957303f7ae299be054b032e30d8f5690f7124c89087739a: Status 404 returned error can't find the container with id bbd2cdda1e07206e3957303f7ae299be054b032e30d8f5690f7124c89087739a
Sep 19 12:58:09.786436 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:09.786385    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" event={"ID":"94654102-6b7c-4781-894a-2973995e77b4","Type":"ContainerStarted","Data":"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"}
Sep 19 12:58:09.786934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:09.786459    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" event={"ID":"94654102-6b7c-4781-894a-2973995e77b4","Type":"ContainerStarted","Data":"bbd2cdda1e07206e3957303f7ae299be054b032e30d8f5690f7124c89087739a"}
Sep 19 12:58:09.786934 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:09.786485    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:09.804210 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:09.804167    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" podStartSLOduration=1.804151777 podStartE2EDuration="1.804151777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:58:09.802400165 +0000 UTC m=+1974.091955933" watchObservedRunningTime="2026-09-19 12:58:09.804151777 +0000 UTC m=+1974.093707545"
Sep 19 12:58:18.193701 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:18.193667    2568 scope.go:117] "RemoveContainer" containerID="9a3c52164ef5eefadb49f52a7de194c00a7a3fdf2e220eda0f6bd066543075ac"
Sep 19 12:58:18.201710 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:18.201684    2568 scope.go:117] "RemoveContainer" containerID="652904d4b82febae629916161c245deb1153583b06d7d1991fc183863be55359"
Sep 19 12:58:20.793664 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:20.793629    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:23.864898 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.864821    2568 generic.go:356] "Generic (PLEG): container finished" podID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerID="4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9" exitCode=0
Sep 19 12:58:23.864898 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.864871    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerDied","Data":"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"}
Sep 19 12:58:23.865327 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.865277    2568 scope.go:117] "RemoveContainer" containerID="4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"
Sep 19 12:58:23.866357 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.866326    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:23.866411 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.866394    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:23.866447 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:23.866425    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:58:24.877436 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:24.876487    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerStarted","Data":"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"}
Sep 19 12:58:25.135686 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:25.135601    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:25.135875 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:25.135841    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 19 12:58:25.135920 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:25.135887    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 19 12:58:26.135451 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:26.135415    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 19 12:58:26.135929 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:26.135480    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 19 12:58:27.134960 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:27.134927    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 19 12:58:27.135142 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:27.134982    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 19 12:58:28.135417 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:28.135377    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 19 12:58:28.135842 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:28.135430    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 19 12:58:29.135812 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:29.135771    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 19 12:58:29.136199 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:29.135842    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 19 12:58:30.139202 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:30.139165    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:30.139593 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:30.139506    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:30.140234 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:30.140218    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:35.573986 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.573949    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"]
Sep 19 12:58:35.577333 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.577316    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:35.579773 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.579754    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:58:35.579883 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.579859    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:58:35.584972 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.584950    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"]
Sep 19 12:58:35.591224 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.591199    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4\" (UID: \"988c5018-dc7e-46b8-9125-a3223b59ec2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:35.691993 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.691961    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4\" (UID: \"988c5018-dc7e-46b8-9125-a3223b59ec2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:35.692611 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.692589    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4\" (UID: \"988c5018-dc7e-46b8-9125-a3223b59ec2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:35.888662 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:35.888625    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:36.161348 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:36.161268    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"]
Sep 19 12:58:36.164557 ip-10-0-136-161 kubenswrapper[2568]: W0919 12:58:36.164526    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988c5018_dc7e_46b8_9125_a3223b59ec2f.slice/crio-4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b WatchSource:0}: Error finding container 4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b: Status 404 returned error can't find the container with id 4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b
Sep 19 12:58:36.949471 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:36.949439    2568 generic.go:356] "Generic (PLEG): container finished" podID="988c5018-dc7e-46b8-9125-a3223b59ec2f" containerID="0a65d0d18b96864efb9c71b1a02d17cf87220267df6e0aa982e82ae0ee64ef32" exitCode=0
Sep 19 12:58:36.949860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:36.949485    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4" event={"ID":"988c5018-dc7e-46b8-9125-a3223b59ec2f","Type":"ContainerDied","Data":"0a65d0d18b96864efb9c71b1a02d17cf87220267df6e0aa982e82ae0ee64ef32"}
Sep 19 12:58:36.949860 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:36.949556    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4" event={"ID":"988c5018-dc7e-46b8-9125-a3223b59ec2f","Type":"ContainerStarted","Data":"4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b"}
Sep 19 12:58:38.091390 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.091362    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:38.111446 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.111417    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca\") pod \"988c5018-dc7e-46b8-9125-a3223b59ec2f\" (UID: \"988c5018-dc7e-46b8-9125-a3223b59ec2f\") "
Sep 19 12:58:38.111761 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.111739    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "988c5018-dc7e-46b8-9125-a3223b59ec2f" (UID: "988c5018-dc7e-46b8-9125-a3223b59ec2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:58:38.212788 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.212712    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988c5018-dc7e-46b8-9125-a3223b59ec2f-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:38.961111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.961080    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4" event={"ID":"988c5018-dc7e-46b8-9125-a3223b59ec2f","Type":"ContainerDied","Data":"4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b"}
Sep 19 12:58:38.961111 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.961105    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"
Sep 19 12:58:38.961330 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:38.961125    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fb8346555f86928c29256e7039700ce1d5f8a90cd485e7f3b0a030129cb6e0b"
Sep 19 12:58:46.508995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.508965    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:46.509442 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.509198    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" podUID="94654102-6b7c-4781-894a-2973995e77b4" containerName="payload-processing" containerID="cri-o://e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c" gracePeriod=30
Sep 19 12:58:46.693319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.693282    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"]
Sep 19 12:58:46.695347 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.695324    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1593f0-jtml4"]
Sep 19 12:58:46.771236 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.771180    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:46.891043 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.891010    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94dm7\" (UniqueName: \"kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7\") pod \"94654102-6b7c-4781-894a-2973995e77b4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") "
Sep 19 12:58:46.891237 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.891079    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume\") pod \"94654102-6b7c-4781-894a-2973995e77b4\" (UID: \"94654102-6b7c-4781-894a-2973995e77b4\") "
Sep 19 12:58:46.891540 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.891474    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "94654102-6b7c-4781-894a-2973995e77b4" (UID: "94654102-6b7c-4781-894a-2973995e77b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:58:46.893155 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.893133    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7" (OuterVolumeSpecName: "kube-api-access-94dm7") pod "94654102-6b7c-4781-894a-2973995e77b4" (UID: "94654102-6b7c-4781-894a-2973995e77b4"). InnerVolumeSpecName "kube-api-access-94dm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:58:46.992538 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.992483    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94654102-6b7c-4781-894a-2973995e77b4-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:46.992538 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:46.992534    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94dm7\" (UniqueName: \"kubernetes.io/projected/94654102-6b7c-4781-894a-2973995e77b4-kube-api-access-94dm7\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:47.001935 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.001901    2568 generic.go:356] "Generic (PLEG): container finished" podID="94654102-6b7c-4781-894a-2973995e77b4" containerID="e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c" exitCode=0
Sep 19 12:58:47.002059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.001935    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" event={"ID":"94654102-6b7c-4781-894a-2973995e77b4","Type":"ContainerDied","Data":"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"}
Sep 19 12:58:47.002059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.001956    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"
Sep 19 12:58:47.002059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.001997    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4" event={"ID":"94654102-6b7c-4781-894a-2973995e77b4","Type":"ContainerDied","Data":"bbd2cdda1e07206e3957303f7ae299be054b032e30d8f5690f7124c89087739a"}
Sep 19 12:58:47.002059 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.002028    2568 scope.go:117] "RemoveContainer" containerID="e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"
Sep 19 12:58:47.010846 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.010832    2568 scope.go:117] "RemoveContainer" containerID="e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"
Sep 19 12:58:47.011114 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:58:47.011096    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c\": container with ID starting with e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c not found: ID does not exist" containerID="e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"
Sep 19 12:58:47.011174 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.011119    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c"} err="failed to get container status \"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c\": rpc error: code = NotFound desc = could not find container \"e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c\": container with ID starting with e85d39587de003ee197cc5203f1e64a4ef5f64444a4cdc80e7e9c1a01f24f71c not found: ID does not exist"
Sep 19 12:58:47.023016 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.022968    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:47.035289 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.035269    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1593f0-c747d9b95-9g9z4"]
Sep 19 12:58:47.721741 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.721712    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94654102-6b7c-4781-894a-2973995e77b4" path="/var/lib/kubelet/pods/94654102-6b7c-4781-894a-2973995e77b4/volumes"
Sep 19 12:58:47.722109 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:47.722028    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988c5018-dc7e-46b8-9125-a3223b59ec2f" path="/var/lib/kubelet/pods/988c5018-dc7e-46b8-9125-a3223b59ec2f/volumes"
Sep 19 12:58:53.469453 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:53.469417    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:53.469949 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:53.469675    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy" containerID="cri-o://645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e" gracePeriod=30
Sep 19 12:58:58.725956 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.725932    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:58.803230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803155    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803191    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kl7b\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803230 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803223    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803247    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803294    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803319    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803488 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803452    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803489    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803544    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data\") pod \"1cf048b2-93cf-4c8d-a12e-d8280881c434\" (UID: \"1cf048b2-93cf-4c8d-a12e-d8280881c434\") "
Sep 19 12:58:58.803750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803621    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:58:58.803750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803668    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:58:58.803750 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803688    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:58:58.804007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803889    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:58:58.804007 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.803983    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:58:58.804117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.804059    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.804117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.804079    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf048b2-93cf-4c8d-a12e-d8280881c434-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.804117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.804094    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.804117 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.804108    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.805585 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.805559    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b" (OuterVolumeSpecName: "kube-api-access-2kl7b") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "kube-api-access-2kl7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:58:58.805698 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.805671    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:58:58.805783 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.805769    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:58:58.805869 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.805853    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cf048b2-93cf-4c8d-a12e-d8280881c434" (UID: "1cf048b2-93cf-4c8d-a12e-d8280881c434"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:58:58.904921 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.904892    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.904921 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.904915    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.904921 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.904925    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kl7b\" (UniqueName: \"kubernetes.io/projected/1cf048b2-93cf-4c8d-a12e-d8280881c434-kube-api-access-2kl7b\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.905124 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.904936    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:58.905124 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:58.904944    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf048b2-93cf-4c8d-a12e-d8280881c434-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:58:59.068777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.068680    2568 generic.go:356] "Generic (PLEG): container finished" podID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerID="645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e" exitCode=0
Sep 19 12:58:59.068777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.068726    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"
Sep 19 12:58:59.068777 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.068756    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerDied","Data":"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"}
Sep 19 12:58:59.069016 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.068816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd" event={"ID":"1cf048b2-93cf-4c8d-a12e-d8280881c434","Type":"ContainerDied","Data":"dd2f0663732ae8c81b23d61ed72b7c054d732424823e06872a7aa7205299b278"}
Sep 19 12:58:59.069016 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.068849    2568 scope.go:117] "RemoveContainer" containerID="645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"
Sep 19 12:58:59.078113 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.078093    2568 scope.go:117] "RemoveContainer" containerID="4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"
Sep 19 12:58:59.086443 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.086423    2568 scope.go:117] "RemoveContainer" containerID="645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"
Sep 19 12:58:59.086707 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:58:59.086690    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e\": container with ID starting with 645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e not found: ID does not exist" containerID="645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"
Sep 19 12:58:59.086758 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.086717    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e"} err="failed to get container status \"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e\": rpc error: code = NotFound desc = could not find container \"645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e\": container with ID starting with 645bd89f845268704bd462e37ac9c1c982a22860bcf0c544eddca72d57db623e not found: ID does not exist"
Sep 19 12:58:59.086758 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.086734    2568 scope.go:117] "RemoveContainer" containerID="4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"
Sep 19 12:58:59.086954 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:58:59.086938    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9\": container with ID starting with 4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9 not found: ID does not exist" containerID="4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"
Sep 19 12:58:59.086995 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.086960    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9"} err="failed to get container status \"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9\": rpc error: code = NotFound desc = could not find container \"4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9\": container with ID starting with 4037a584a7a22ad8d362acc31d843efa04f96cbce8f074140d56c44f822196a9 not found: ID does not exist"
Sep 19 12:58:59.091058 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.091036    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:59.095298 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.095278    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1593f0-openshift-default-c5cf7769b-dkhcd"]
Sep 19 12:58:59.722755 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:58:59.722722    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" path="/var/lib/kubelet/pods/1cf048b2-93cf-4c8d-a12e-d8280881c434/volumes"
Sep 19 12:59:12.564410 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:12.564374    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:59:12.564864 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:12.564637    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" podUID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" containerName="payload-pre-processing" containerID="cri-o://caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f" gracePeriod=30
Sep 19 12:59:12.857383 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:12.857351    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:59:12.944457 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:12.944428    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume\") pod \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") "
Sep 19 12:59:12.944862 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:12.944835    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35f0555d-3ee9-4606-8b5c-03507cd9dc26" (UID: "35f0555d-3ee9-4606-8b5c-03507cd9dc26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:59:13.045713 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.045677    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfqmb\" (UniqueName: \"kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb\") pod \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\" (UID: \"35f0555d-3ee9-4606-8b5c-03507cd9dc26\") "
Sep 19 12:59:13.045892 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.045880    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0555d-3ee9-4606-8b5c-03507cd9dc26-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:59:13.047738 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.047712    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb" (OuterVolumeSpecName: "kube-api-access-kfqmb") pod "35f0555d-3ee9-4606-8b5c-03507cd9dc26" (UID: "35f0555d-3ee9-4606-8b5c-03507cd9dc26"). InnerVolumeSpecName "kube-api-access-kfqmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:59:13.139708 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.139676    2568 generic.go:356] "Generic (PLEG): container finished" podID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" containerID="caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f" exitCode=0
Sep 19 12:59:13.139868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.139715    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"
Sep 19 12:59:13.139868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.139755    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" event={"ID":"35f0555d-3ee9-4606-8b5c-03507cd9dc26","Type":"ContainerDied","Data":"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"}
Sep 19 12:59:13.139868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.139806    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp" event={"ID":"35f0555d-3ee9-4606-8b5c-03507cd9dc26","Type":"ContainerDied","Data":"fea0794f12a782cf26f702431faba3059e607034e21548c36df8a988680794bd"}
Sep 19 12:59:13.139868 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.139828    2568 scope.go:117] "RemoveContainer" containerID="caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"
Sep 19 12:59:13.146319 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.146301    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfqmb\" (UniqueName: \"kubernetes.io/projected/35f0555d-3ee9-4606-8b5c-03507cd9dc26-kube-api-access-kfqmb\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 12:59:13.149337 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.149320    2568 scope.go:117] "RemoveContainer" containerID="caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"
Sep 19 12:59:13.149626 ip-10-0-136-161 kubenswrapper[2568]: E0919 12:59:13.149606    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f\": container with ID starting with caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f not found: ID does not exist" containerID="caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"
Sep 19 12:59:13.149690 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.149639    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f"} err="failed to get container status \"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f\": rpc error: code = NotFound desc = could not find container \"caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f\": container with ID starting with caaa8304a7a4b53745b93f187f3b4b198341cc77578d1ee82c14f752eb4d659f not found: ID does not exist"
Sep 19 12:59:13.162435 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.162415    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:59:13.166922 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.166901    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f32129-668746f8b6-t8bzp"]
Sep 19 12:59:13.722335 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:13.722305    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" path="/var/lib/kubelet/pods/35f0555d-3ee9-4606-8b5c-03507cd9dc26/volumes"
Sep 19 12:59:18.225570 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:18.225537    2568 scope.go:117] "RemoveContainer" containerID="6831bf4829cfaac5dc5ad3c23a88990db9a51c93a10cc188a377e12b8b9e7068"
Sep 19 12:59:18.233709 ip-10-0-136-161 kubenswrapper[2568]: I0919 12:59:18.233692    2568 scope.go:117] "RemoveContainer" containerID="0e99bed13cd3d63f184aa0e48d43237b1adcf7fb83b92b902f3db3ceae636dcd"
Sep 19 13:00:00.146388 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146353    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"]
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146772    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94654102-6b7c-4781-894a-2973995e77b4" containerName="payload-processing"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146785    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="94654102-6b7c-4781-894a-2973995e77b4" containerName="payload-processing"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146794    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146800    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146806    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988c5018-dc7e-46b8-9125-a3223b59ec2f" containerName="revoke-keys"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146811    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="988c5018-dc7e-46b8-9125-a3223b59ec2f" containerName="revoke-keys"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146819    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146825    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146833    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" containerName="payload-pre-processing"
Sep 19 13:00:00.146848 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146838    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" containerName="payload-pre-processing"
Sep 19 13:00:00.147486 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146891    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="94654102-6b7c-4781-894a-2973995e77b4" containerName="payload-processing"
Sep 19 13:00:00.147486 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146899    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.147486 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146904    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="988c5018-dc7e-46b8-9125-a3223b59ec2f" containerName="revoke-keys"
Sep 19 13:00:00.147486 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146914    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="35f0555d-3ee9-4606-8b5c-03507cd9dc26" containerName="payload-pre-processing"
Sep 19 13:00:00.147486 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.146919    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf048b2-93cf-4c8d-a12e-d8280881c434" containerName="istio-proxy"
Sep 19 13:00:00.149873 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.149856    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"]
Sep 19 13:00:00.149987 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.149967    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"
Sep 19 13:00:00.152711 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.152691    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 13:00:00.153184 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.153171    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"
Sep 19 13:00:00.159378 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.159351    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"]
Sep 19 13:00:00.161433 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.161412    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"]
Sep 19 13:00:00.163211 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.163194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"
Sep 19 13:00:00.171815 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.171787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"
Sep 19 13:00:00.463297 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.463273    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"]
Sep 19 13:00:00.465374 ip-10-0-136-161 kubenswrapper[2568]: W0919 13:00:00.465340    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67670be_b884_4dbe_8839_dc51c3cb847a.slice/crio-a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770 WatchSource:0}: Error finding container a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770: Status 404 returned error can't find the container with id a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770
Sep 19 13:00:00.465662 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:00.465639    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"]
Sep 19 13:00:00.466277 ip-10-0-136-161 kubenswrapper[2568]: W0919 13:00:00.466249    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3dfd7b_2768_4b26_b705_d6f3f1ca1eae.slice/crio-bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb WatchSource:0}: Error finding container bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb: Status 404 returned error can't find the container with id bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb
Sep 19 13:00:01.387783 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:01.387756    2568 generic.go:356] "Generic (PLEG): container finished" podID="d67670be-b884-4dbe-8839-dc51c3cb847a" containerID="20a1909fd7586c828dd0a92b601788625c824efce0bc114711f6fba14624e9ae" exitCode=0
Sep 19 13:00:01.388200 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:01.387809    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm" event={"ID":"d67670be-b884-4dbe-8839-dc51c3cb847a","Type":"ContainerDied","Data":"20a1909fd7586c828dd0a92b601788625c824efce0bc114711f6fba14624e9ae"}
Sep 19 13:00:01.388200 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:01.387870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm" event={"ID":"d67670be-b884-4dbe-8839-dc51c3cb847a","Type":"ContainerStarted","Data":"a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770"}
Sep 19 13:00:01.389339 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:01.389319    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5" event={"ID":"5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae","Type":"ContainerStarted","Data":"bbec4de5e8a1452b5e3266362e4c28ced43a87eb51d729dfa8512d5c35236ea9"}
Sep 19 13:00:01.389431 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:01.389353    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5" event={"ID":"5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae","Type":"ContainerStarted","Data":"bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb"}
Sep 19 13:00:02.157318 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.157289    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"]
Sep 19 13:00:02.160773 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.160754    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:02.165275 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.165253    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 13:00:02.172817 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.172794    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"]
Sep 19 13:00:02.199351 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.199317    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr\" (UID: \"761cbd52-3f25-4463-a57a-35870b747b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:02.299986 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.299949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr\" (UID: \"761cbd52-3f25-4463-a57a-35870b747b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:02.300591 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.300572    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr\" (UID: \"761cbd52-3f25-4463-a57a-35870b747b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:02.395673 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.395614    2568 generic.go:356] "Generic (PLEG): container finished" podID="5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae" containerID="bbec4de5e8a1452b5e3266362e4c28ced43a87eb51d729dfa8512d5c35236ea9" exitCode=0
Sep 19 13:00:02.396091 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.395697    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5" event={"ID":"5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae","Type":"ContainerDied","Data":"bbec4de5e8a1452b5e3266362e4c28ced43a87eb51d729dfa8512d5c35236ea9"}
Sep 19 13:00:02.470862 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.470832    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:02.572789 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.572753    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"
Sep 19 13:00:02.575266 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.575240    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"
Sep 19 13:00:02.783146 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:02.783003    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"]
Sep 19 13:00:02.785950 ip-10-0-136-161 kubenswrapper[2568]: W0919 13:00:02.785923    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761cbd52_3f25_4463_a57a_35870b747b61.slice/crio-46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9 WatchSource:0}: Error finding container 46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9: Status 404 returned error can't find the container with id 46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9
Sep 19 13:00:03.402307 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.402275    2568 generic.go:356] "Generic (PLEG): container finished" podID="761cbd52-3f25-4463-a57a-35870b747b61" containerID="225d527b8a6d7e9fde22ad5bd653dfc244947f1d6e5838bfa872893ef0ba2793" exitCode=0
Sep 19 13:00:03.402751 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.402358    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr" event={"ID":"761cbd52-3f25-4463-a57a-35870b747b61","Type":"ContainerDied","Data":"225d527b8a6d7e9fde22ad5bd653dfc244947f1d6e5838bfa872893ef0ba2793"}
Sep 19 13:00:03.402751 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.402432    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr" event={"ID":"761cbd52-3f25-4463-a57a-35870b747b61","Type":"ContainerStarted","Data":"46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9"}
Sep 19 13:00:03.403636 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.403617    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"
Sep 19 13:00:03.403759 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.403657    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5" event={"ID":"5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae","Type":"ContainerDied","Data":"bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb"}
Sep 19 13:00:03.403759 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.403702    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb8e9472bec8b8d1f302131d33a57667c38d3b8d7a10d4e82075dd0ef9a435bb"
Sep 19 13:00:03.404871 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.404849    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm" event={"ID":"d67670be-b884-4dbe-8839-dc51c3cb847a","Type":"ContainerDied","Data":"a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770"}
Sep 19 13:00:03.404965 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.404879    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"
Sep 19 13:00:03.404965 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:03.404889    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4885a0e16cecf0f4960978a67a06b6339f4805c0aacbc03a56decf9f1a89770"
Sep 19 13:00:04.544307 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:04.544281    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:04.620053 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:04.620019    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca\") pod \"761cbd52-3f25-4463-a57a-35870b747b61\" (UID: \"761cbd52-3f25-4463-a57a-35870b747b61\") "
Sep 19 13:00:04.620415 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:04.620393    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "761cbd52-3f25-4463-a57a-35870b747b61" (UID: "761cbd52-3f25-4463-a57a-35870b747b61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:04.721446 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:04.721373    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/761cbd52-3f25-4463-a57a-35870b747b61-maas-api-service-ca\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:05.416965 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:05.416934    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"
Sep 19 13:00:05.417147 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:05.416969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr" event={"ID":"761cbd52-3f25-4463-a57a-35870b747b61","Type":"ContainerDied","Data":"46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9"}
Sep 19 13:00:05.417147 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:05.417011    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46b27de8e6cb2c947a7c139caa1796462a54d08f6e0f288dab03d8cd838a87c9"
Sep 19 13:00:12.774910 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.774874    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"]
Sep 19 13:00:12.778270 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.778248    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-4e754a-29830380-8zkgm"]
Sep 19 13:00:12.811392 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.811361    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 13:00:12.811619 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.811569    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" podUID="56e04786-492c-4fc7-a065-0e8e27c1648e" containerName="payload-pre-processing" containerID="cri-o://28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa" gracePeriod=30
Sep 19 13:00:12.966056 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.966006    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"]
Sep 19 13:00:12.968433 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:12.968407    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-4e754a-bjndr"]
Sep 19 13:00:13.063405 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.063378    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 13:00:13.201463 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.201429    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume\") pod \"56e04786-492c-4fc7-a065-0e8e27c1648e\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") "
Sep 19 13:00:13.201678 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.201547    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlmf9\" (UniqueName: \"kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9\") pod \"56e04786-492c-4fc7-a065-0e8e27c1648e\" (UID: \"56e04786-492c-4fc7-a065-0e8e27c1648e\") "
Sep 19 13:00:13.201816 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.201792    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56e04786-492c-4fc7-a065-0e8e27c1648e" (UID: "56e04786-492c-4fc7-a065-0e8e27c1648e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:13.203589 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.203565    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9" (OuterVolumeSpecName: "kube-api-access-tlmf9") pod "56e04786-492c-4fc7-a065-0e8e27c1648e" (UID: "56e04786-492c-4fc7-a065-0e8e27c1648e"). InnerVolumeSpecName "kube-api-access-tlmf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:13.303101 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.303019    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlmf9\" (UniqueName: \"kubernetes.io/projected/56e04786-492c-4fc7-a065-0e8e27c1648e-kube-api-access-tlmf9\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:13.303101 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.303060    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e04786-492c-4fc7-a065-0e8e27c1648e-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:13.458213 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.458173    2568 generic.go:356] "Generic (PLEG): container finished" podID="56e04786-492c-4fc7-a065-0e8e27c1648e" containerID="28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa" exitCode=0
Sep 19 13:00:13.458358 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.458235    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"
Sep 19 13:00:13.458358 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.458254    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" event={"ID":"56e04786-492c-4fc7-a065-0e8e27c1648e","Type":"ContainerDied","Data":"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"}
Sep 19 13:00:13.458358 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.458341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf" event={"ID":"56e04786-492c-4fc7-a065-0e8e27c1648e","Type":"ContainerDied","Data":"077124c3b4f714a47c19b9b1b2457cc4b0a3a8835bade9fd396a59ddf517fe7e"}
Sep 19 13:00:13.458483 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.458364    2568 scope.go:117] "RemoveContainer" containerID="28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"
Sep 19 13:00:13.467264 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.467248    2568 scope.go:117] "RemoveContainer" containerID="28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"
Sep 19 13:00:13.467525 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:13.467489    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa\": container with ID starting with 28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa not found: ID does not exist" containerID="28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"
Sep 19 13:00:13.467599 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.467529    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa"} err="failed to get container status \"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa\": rpc error: code = NotFound desc = could not find container \"28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa\": container with ID starting with 28b6e8dcbe6e6afa0e44fe01e74eda80ea49718d44083cb9a2e5357eef9913fa not found: ID does not exist"
Sep 19 13:00:13.479454 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.479431    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 13:00:13.481153 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.481124    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4e754a-bf47f64b6-mq5sf"]
Sep 19 13:00:13.722060 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.722028    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e04786-492c-4fc7-a065-0e8e27c1648e" path="/var/lib/kubelet/pods/56e04786-492c-4fc7-a065-0e8e27c1648e/volumes"
Sep 19 13:00:13.722441 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.722422    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761cbd52-3f25-4463-a57a-35870b747b61" path="/var/lib/kubelet/pods/761cbd52-3f25-4463-a57a-35870b747b61/volumes"
Sep 19 13:00:13.722766 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:13.722749    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67670be-b884-4dbe-8839-dc51c3cb847a" path="/var/lib/kubelet/pods/d67670be-b884-4dbe-8839-dc51c3cb847a/volumes"
Sep 19 13:00:17.801606 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:17.801579    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 13:00:17.804976 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:17.804955    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 13:00:18.267149 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:18.267116    2568 scope.go:117] "RemoveContainer" containerID="f234797ab386a9b2abd536828953cf2bef4c71cae87796d49da2f12dbbbdf13e"
Sep 19 13:00:30.839017 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:30.838979    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 13:00:30.839686 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:30.839227    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" podUID="a868abf8-beb6-4785-b51b-0701b4628feb" containerName="payload-processing" containerID="cri-o://66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518" gracePeriod=30
Sep 19 13:00:30.844410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:30.844379    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 13:00:30.844689 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:30.844654    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" podUID="414a4cfb-0671-4081-b387-ddf45268f021" containerName="payload-pre-processing" containerID="cri-o://662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8" gracePeriod=30
Sep 19 13:00:31.112510 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.112475    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 13:00:31.115077 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.115059    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 13:00:31.278378 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278328    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r69h4\" (UniqueName: \"kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4\") pod \"414a4cfb-0671-4081-b387-ddf45268f021\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") "
Sep 19 13:00:31.278378 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278381    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume\") pod \"414a4cfb-0671-4081-b387-ddf45268f021\" (UID: \"414a4cfb-0671-4081-b387-ddf45268f021\") "
Sep 19 13:00:31.278677 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278489    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjxlf\" (UniqueName: \"kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf\") pod \"a868abf8-beb6-4785-b51b-0701b4628feb\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") "
Sep 19 13:00:31.278677 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278608    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume\") pod \"a868abf8-beb6-4785-b51b-0701b4628feb\" (UID: \"a868abf8-beb6-4785-b51b-0701b4628feb\") "
Sep 19 13:00:31.278814 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278783    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "414a4cfb-0671-4081-b387-ddf45268f021" (UID: "414a4cfb-0671-4081-b387-ddf45268f021"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:31.278997 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278972    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a868abf8-beb6-4785-b51b-0701b4628feb" (UID: "a868abf8-beb6-4785-b51b-0701b4628feb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:31.279053 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.278986    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a4cfb-0671-4081-b387-ddf45268f021-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:31.280695 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.280667    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf" (OuterVolumeSpecName: "kube-api-access-jjxlf") pod "a868abf8-beb6-4785-b51b-0701b4628feb" (UID: "a868abf8-beb6-4785-b51b-0701b4628feb"). InnerVolumeSpecName "kube-api-access-jjxlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:31.280695 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.280690    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4" (OuterVolumeSpecName: "kube-api-access-r69h4") pod "414a4cfb-0671-4081-b387-ddf45268f021" (UID: "414a4cfb-0671-4081-b387-ddf45268f021"). InnerVolumeSpecName "kube-api-access-r69h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:31.380203 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.380174    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r69h4\" (UniqueName: \"kubernetes.io/projected/414a4cfb-0671-4081-b387-ddf45268f021-kube-api-access-r69h4\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:31.380203 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.380204    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjxlf\" (UniqueName: \"kubernetes.io/projected/a868abf8-beb6-4785-b51b-0701b4628feb-kube-api-access-jjxlf\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:31.380390 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.380214    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a868abf8-beb6-4785-b51b-0701b4628feb-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:31.548992 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.548957    2568 generic.go:356] "Generic (PLEG): container finished" podID="a868abf8-beb6-4785-b51b-0701b4628feb" containerID="66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518" exitCode=0
Sep 19 13:00:31.549177 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.549005    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"
Sep 19 13:00:31.549177 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.549043    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" event={"ID":"a868abf8-beb6-4785-b51b-0701b4628feb","Type":"ContainerDied","Data":"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"}
Sep 19 13:00:31.549177 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.549135    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w" event={"ID":"a868abf8-beb6-4785-b51b-0701b4628feb","Type":"ContainerDied","Data":"c2ffa25ac50fe15f969a7ff686be07760725fc43b383564bb34d43c5e741e153"}
Sep 19 13:00:31.549177 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.549171    2568 scope.go:117] "RemoveContainer" containerID="66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"
Sep 19 13:00:31.550457 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.550441    2568 generic.go:356] "Generic (PLEG): container finished" podID="414a4cfb-0671-4081-b387-ddf45268f021" containerID="662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8" exitCode=0
Sep 19 13:00:31.550555 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.550489    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"
Sep 19 13:00:31.550555 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.550529    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" event={"ID":"414a4cfb-0671-4081-b387-ddf45268f021","Type":"ContainerDied","Data":"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"}
Sep 19 13:00:31.550635 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.550576    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj" event={"ID":"414a4cfb-0671-4081-b387-ddf45268f021","Type":"ContainerDied","Data":"c8b2904ad384c55e6824f9908a5171acf3ac9ce76905874855445de333945ec7"}
Sep 19 13:00:31.558444 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.558318    2568 scope.go:117] "RemoveContainer" containerID="66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"
Sep 19 13:00:31.558653 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:31.558632    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518\": container with ID starting with 66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518 not found: ID does not exist" containerID="66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"
Sep 19 13:00:31.558722 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.558665    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518"} err="failed to get container status \"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518\": rpc error: code = NotFound desc = could not find container \"66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518\": container with ID starting with 66148a0899ca11f108a2cdbd5c88c12c8105716f9ac486a2b7f54820ab4ca518 not found: ID does not exist"
Sep 19 13:00:31.558722 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.558688    2568 scope.go:117] "RemoveContainer" containerID="662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"
Sep 19 13:00:31.567234 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.567216    2568 scope.go:117] "RemoveContainer" containerID="662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"
Sep 19 13:00:31.567523 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:31.567485    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8\": container with ID starting with 662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8 not found: ID does not exist" containerID="662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"
Sep 19 13:00:31.567609 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.567532    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8"} err="failed to get container status \"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8\": rpc error: code = NotFound desc = could not find container \"662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8\": container with ID starting with 662d966b956517d6ecbfc995618529c594e34f3d3b77f595b4648bddfac1d6f8 not found: ID does not exist"
Sep 19 13:00:31.575546 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.575523    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 13:00:31.581124 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.581102    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b0e30e-bddd54478-shjhj"]
Sep 19 13:00:31.591544 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.591522    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 13:00:31.595143 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.595123    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b0e30e-857b6887df-49v7w"]
Sep 19 13:00:31.722088 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.722010    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="414a4cfb-0671-4081-b387-ddf45268f021" path="/var/lib/kubelet/pods/414a4cfb-0671-4081-b387-ddf45268f021/volumes"
Sep 19 13:00:31.722368 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:31.722356    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a868abf8-beb6-4785-b51b-0701b4628feb" path="/var/lib/kubelet/pods/a868abf8-beb6-4785-b51b-0701b4628feb/volumes"
Sep 19 13:00:37.868487 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:37.868452    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 13:00:37.868963 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:37.868730    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy" containerID="cri-o://d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc" gracePeriod=30
Sep 19 13:00:43.110022 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.109999    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 13:00:43.181145 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181071    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181145 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181109    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181145 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181136    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr959\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181164    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181189    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181234    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181270    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181384    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:43.181410 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181391    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data" (OuterVolumeSpecName: "istio-data") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:43.181829 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181426    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181829 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181519    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token\") pod \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\" (UID: \"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390\") "
Sep 19 13:00:43.181829 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181550    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:43.181829 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181639    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:43.181829 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181675    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:43.182095 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181880    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.182095 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181900    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.182095 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181915    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.182095 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181929    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.182095 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.181943    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.183554 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.183524    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:43.183554 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.183525    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token" (OuterVolumeSpecName: "istio-token") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:43.183710 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.183556    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959" (OuterVolumeSpecName: "kube-api-access-dr959") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "kube-api-access-dr959". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:43.183710 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.183575    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" (UID: "b361b4d4-bb4a-47e5-83a5-bd64cf0f6390"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 13:00:43.282399 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.282361    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.282399 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.282393    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.282399 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.282405    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.282669 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.282417    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr959\" (UniqueName: \"kubernetes.io/projected/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390-kube-api-access-dr959\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:43.610091 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.609991    2568 generic.go:356] "Generic (PLEG): container finished" podID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerID="d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc" exitCode=0
Sep 19 13:00:43.610091 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.610055    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"
Sep 19 13:00:43.610091 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.610073    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerDied","Data":"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"}
Sep 19 13:00:43.610336 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.610160    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z" event={"ID":"b361b4d4-bb4a-47e5-83a5-bd64cf0f6390","Type":"ContainerDied","Data":"da4c4a56cccb77c67667876eb779911e7ca05bd9764a4bc142869aad5efbebc1"}
Sep 19 13:00:43.610336 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.610186    2568 scope.go:117] "RemoveContainer" containerID="d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"
Sep 19 13:00:43.620363 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.620345    2568 scope.go:117] "RemoveContainer" containerID="63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"
Sep 19 13:00:43.628633 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.628615    2568 scope.go:117] "RemoveContainer" containerID="d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"
Sep 19 13:00:43.628875 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:43.628857    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc\": container with ID starting with d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc not found: ID does not exist" containerID="d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"
Sep 19 13:00:43.628943 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.628887    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc"} err="failed to get container status \"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc\": rpc error: code = NotFound desc = could not find container \"d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc\": container with ID starting with d6dd190d67707063097c3667a6ab3dbc834883d4bf59371dcf298b97222f30fc not found: ID does not exist"
Sep 19 13:00:43.628943 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.628911    2568 scope.go:117] "RemoveContainer" containerID="63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"
Sep 19 13:00:43.629158 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:43.629143    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0\": container with ID starting with 63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0 not found: ID does not exist" containerID="63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"
Sep 19 13:00:43.629215 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.629165    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0"} err="failed to get container status \"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0\": rpc error: code = NotFound desc = could not find container \"63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0\": container with ID starting with 63d807a7a5b614c9ded3479f9b500b6d44c0169479720a11cd8366befe6fc6d0 not found: ID does not exist"
Sep 19 13:00:43.629999 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:43.629981    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb361b4d4_bb4a_47e5_83a5_bd64cf0f6390.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 13:00:43.634106 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.634083    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 13:00:43.638066 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.638045    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b0e30e-openshift-default-5f8c8b5c66-mmh5z"]
Sep 19 13:00:43.721729 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:43.721704    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" path="/var/lib/kubelet/pods/b361b4d4-bb4a-47e5-83a5-bd64cf0f6390/volumes"
Sep 19 13:00:48.777511 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:48.777464    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"]
Sep 19 13:00:48.786244 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:48.786216    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-5d6124-29830380-8bvf5"]
Sep 19 13:00:48.803052 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:48.803026    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 13:00:48.803321 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:48.803281    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" podUID="77c1f288-ac78-4adc-beb4-1d7605102ede" containerName="payload-processing" containerID="cri-o://c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82" gracePeriod=30
Sep 19 13:00:49.051471 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.051441    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 13:00:49.132977 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.132942    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dzk7\" (UniqueName: \"kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7\") pod \"77c1f288-ac78-4adc-beb4-1d7605102ede\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") "
Sep 19 13:00:49.133175 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.133000    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume\") pod \"77c1f288-ac78-4adc-beb4-1d7605102ede\" (UID: \"77c1f288-ac78-4adc-beb4-1d7605102ede\") "
Sep 19 13:00:49.133361 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.133335    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77c1f288-ac78-4adc-beb4-1d7605102ede" (UID: "77c1f288-ac78-4adc-beb4-1d7605102ede"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:49.135017 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.134992    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7" (OuterVolumeSpecName: "kube-api-access-9dzk7") pod "77c1f288-ac78-4adc-beb4-1d7605102ede" (UID: "77c1f288-ac78-4adc-beb4-1d7605102ede"). InnerVolumeSpecName "kube-api-access-9dzk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:49.233960 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.233931    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dzk7\" (UniqueName: \"kubernetes.io/projected/77c1f288-ac78-4adc-beb4-1d7605102ede-kube-api-access-9dzk7\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:49.233960 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.233955    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77c1f288-ac78-4adc-beb4-1d7605102ede-plugins-config-volume\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:00:49.639403 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.639366    2568 generic.go:356] "Generic (PLEG): container finished" podID="77c1f288-ac78-4adc-beb4-1d7605102ede" containerID="c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82" exitCode=0
Sep 19 13:00:49.639591 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.639406    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" event={"ID":"77c1f288-ac78-4adc-beb4-1d7605102ede","Type":"ContainerDied","Data":"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"}
Sep 19 13:00:49.639591 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.639461    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp" event={"ID":"77c1f288-ac78-4adc-beb4-1d7605102ede","Type":"ContainerDied","Data":"37dcc3c3c60d8e4bfb90e4f06e262c5b24a0bcc6fa6d537dc18fd987b9185ad1"}
Sep 19 13:00:49.639591 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.639484    2568 scope.go:117] "RemoveContainer" containerID="c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"
Sep 19 13:00:49.639591 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.639413    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"
Sep 19 13:00:49.648391 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.648374    2568 scope.go:117] "RemoveContainer" containerID="c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"
Sep 19 13:00:49.648686 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:00:49.648667    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82\": container with ID starting with c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82 not found: ID does not exist" containerID="c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"
Sep 19 13:00:49.648764 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.648705    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82"} err="failed to get container status \"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82\": rpc error: code = NotFound desc = could not find container \"c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82\": container with ID starting with c7b04625274a9220a7ff8f8259e488ce5e08405d77af940bcccee36e5107fd82 not found: ID does not exist"
Sep 19 13:00:49.668475 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.668447    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 13:00:49.674440 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.674418    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-5d6124-57c9fcd65c-g6nrp"]
Sep 19 13:00:49.721270 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.721245    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae" path="/var/lib/kubelet/pods/5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae/volumes"
Sep 19 13:00:49.721618 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:49.721605    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c1f288-ac78-4adc-beb4-1d7605102ede" path="/var/lib/kubelet/pods/77c1f288-ac78-4adc-beb4-1d7605102ede/volumes"
Sep 19 13:00:55.446926 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:55.446886    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 13:00:55.447313 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:00:55.447099    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy" containerID="cri-o://62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9" gracePeriod=30
Sep 19 13:01:00.695438 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.695401    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 13:01:00.696289 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.696267    2568 generic.go:356] "Generic (PLEG): container finished" podID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerID="62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9" exitCode=0
Sep 19 13:01:00.696372 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.696327    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerDied","Data":"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"}
Sep 19 13:01:00.696372 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.696368    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv" event={"ID":"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd","Type":"ContainerDied","Data":"88117753b2af7dd00bca06e9c9112b7087aa0e65de2f1f55753474b5fb3bcf1d"}
Sep 19 13:01:00.696445 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.696389    2568 scope.go:117] "RemoveContainer" containerID="62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"
Sep 19 13:01:00.704796 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.704779    2568 scope.go:117] "RemoveContainer" containerID="30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"
Sep 19 13:01:00.717644 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.714122    2568 scope.go:117] "RemoveContainer" containerID="62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"
Sep 19 13:01:00.718189 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:01:00.718033    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9\": container with ID starting with 62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9 not found: ID does not exist" containerID="62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"
Sep 19 13:01:00.718189 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.718073    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9"} err="failed to get container status \"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9\": rpc error: code = NotFound desc = could not find container \"62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9\": container with ID starting with 62b695379f5114a520efd558b6c7f4548bc3c93c7579941b9f3860a2ac27dfc9 not found: ID does not exist"
Sep 19 13:01:00.718189 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.718096    2568 scope.go:117] "RemoveContainer" containerID="30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"
Sep 19 13:01:00.718483 ip-10-0-136-161 kubenswrapper[2568]: E0919 13:01:00.718463    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729\": container with ID starting with 30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729 not found: ID does not exist" containerID="30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"
Sep 19 13:01:00.718639 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.718490    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729"} err="failed to get container status \"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729\": rpc error: code = NotFound desc = could not find container \"30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729\": container with ID starting with 30c1c5f2bef2403813f88b35121d72a25bf82238f7a14d705ed6124122511729 not found: ID does not exist"
Sep 19 13:01:00.739871 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.739837    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740036 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.739906    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740036 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.739979    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740036 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740007    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740041    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740081    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740108    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nklt\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740131    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740155    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740212 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740183    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket\") pod \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\" (UID: \"02ae1836-8a3a-46ff-8f37-6bf2ad5182dd\") "
Sep 19 13:01:00.740529 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740450    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-credential-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.740598 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740562    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:01:00.740598 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740580    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:01:00.740760 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740629    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:01:00.740760 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.740672    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:01:00.742746 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.742713    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:01:00.742926 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.742901    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:01:00.743019 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.742919    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 13:01:00.743019 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.742925    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt" (OuterVolumeSpecName: "kube-api-access-8nklt") pod "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" (UID: "02ae1836-8a3a-46ff-8f37-6bf2ad5182dd"). InnerVolumeSpecName "kube-api-access-8nklt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:01:00.841161 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841131    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-data\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841161 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841156    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-socket\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841161 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841166    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-token\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841393 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841174    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-podinfo\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841393 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841183    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istio-envoy\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841393 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841190    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-istiod-ca-cert\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841393 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841197    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-workload-certs\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:00.841393 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:00.841206    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nklt\" (UniqueName: \"kubernetes.io/projected/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd-kube-api-access-8nklt\") on node \"ip-10-0-136-161.ec2.internal\" DevicePath \"\""
Sep 19 13:01:01.701710 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:01.701669    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"
Sep 19 13:01:01.722793 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:01.722762    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 13:01:01.726556 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:01.726532    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5d6124-openshift-default-b4b4969c8-6b9jv"]
Sep 19 13:01:03.721355 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:01:03.721319    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" path="/var/lib/kubelet/pods/02ae1836-8a3a-46ff-8f37-6bf2ad5182dd/volumes"
Sep 19 13:05:17.828919 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:05:17.828889    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 13:05:17.832901 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:05:17.832873    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nhggd_b122ea00-99d3-47a3-b170-cf888a0ff9af/ovn-acl-logging/0.log"
Sep 19 13:05:18.339028 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:05:18.339000    2568 scope.go:117] "RemoveContainer" containerID="0a65d0d18b96864efb9c71b1a02d17cf87220267df6e0aa982e82ae0ee64ef32"
Sep 19 13:06:18.359227 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:18.359195    2568 scope.go:117] "RemoveContainer" containerID="bbec4de5e8a1452b5e3266362e4c28ced43a87eb51d729dfa8512d5c35236ea9"
Sep 19 13:06:18.368107 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:18.368088    2568 scope.go:117] "RemoveContainer" containerID="225d527b8a6d7e9fde22ad5bd653dfc244947f1d6e5838bfa872893ef0ba2793"
Sep 19 13:06:18.376640 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:18.376622    2568 scope.go:117] "RemoveContainer" containerID="20a1909fd7586c828dd0a92b601788625c824efce0bc114711f6fba14624e9ae"
Sep 19 13:06:42.707901 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.707865    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"]
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708192    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708203    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708215    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c1f288-ac78-4adc-beb4-1d7605102ede" containerName="payload-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708221    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c1f288-ac78-4adc-beb4-1d7605102ede" containerName="payload-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708232    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e04786-492c-4fc7-a065-0e8e27c1648e" containerName="payload-pre-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708239    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e04786-492c-4fc7-a065-0e8e27c1648e" containerName="payload-pre-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708247    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414a4cfb-0671-4081-b387-ddf45268f021" containerName="payload-pre-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708252    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="414a4cfb-0671-4081-b387-ddf45268f021" containerName="payload-pre-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708260    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d67670be-b884-4dbe-8839-dc51c3cb847a" containerName="cleanup"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708265    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67670be-b884-4dbe-8839-dc51c3cb847a" containerName="cleanup"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708272    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708277    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708282    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761cbd52-3f25-4463-a57a-35870b747b61" containerName="revoke-keys"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708287    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="761cbd52-3f25-4463-a57a-35870b747b61" containerName="revoke-keys"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708293    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708298    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708305    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a868abf8-beb6-4785-b51b-0701b4628feb" containerName="payload-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708310    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a868abf8-beb6-4785-b51b-0701b4628feb" containerName="payload-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708319    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae" containerName="cleanup"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708325    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae" containerName="cleanup"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708374    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c1f288-ac78-4adc-beb4-1d7605102ede" containerName="payload-processing"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708381    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d67670be-b884-4dbe-8839-dc51c3cb847a" containerName="cleanup"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708387    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:06:42.708382 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708394    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708399    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e04786-492c-4fc7-a065-0e8e27c1648e" containerName="payload-pre-processing"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708404    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a868abf8-beb6-4785-b51b-0701b4628feb" containerName="payload-processing"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708409    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="761cbd52-3f25-4463-a57a-35870b747b61" containerName="revoke-keys"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708414    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3dfd7b-2768-4b26-b705-d6f3f1ca1eae" containerName="cleanup"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708419    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:06:42.709154 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.708426    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="414a4cfb-0671-4081-b387-ddf45268f021" containerName="payload-pre-processing"
Sep 19 13:06:42.713092 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.713074    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.716652 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.716618    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 13:06:42.716810 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.716720    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 13:06:42.716810 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.716778    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 13:06:42.716810 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.716779    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sr888\""
Sep 19 13:06:42.716810 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.716785    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 13:06:42.723364 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.723299    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"]
Sep 19 13:06:42.801980 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.801946    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-maas-api-tls\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.802195 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.801996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20956a1e-47bb-4e89-9731-166265101190-openshift-service-ca\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.802195 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.802020    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m488b\" (UniqueName: \"kubernetes.io/projected/20956a1e-47bb-4e89-9731-166265101190-kube-api-access-m488b\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.802195 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.802086    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-metrics-certs\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.902628 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.902593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-maas-api-tls\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.902824 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.902640    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20956a1e-47bb-4e89-9731-166265101190-openshift-service-ca\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.902824 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.902661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m488b\" (UniqueName: \"kubernetes.io/projected/20956a1e-47bb-4e89-9731-166265101190-kube-api-access-m488b\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.902824 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.902688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-metrics-certs\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.903520 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.903469    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20956a1e-47bb-4e89-9731-166265101190-openshift-service-ca\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.905081 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.905058    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-metrics-certs\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.905171 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.905095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20956a1e-47bb-4e89-9731-166265101190-maas-api-tls\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:42.909802 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:42.909773    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m488b\" (UniqueName: \"kubernetes.io/projected/20956a1e-47bb-4e89-9731-166265101190-kube-api-access-m488b\") pod \"maas-api-68c86dfd56-7gfwf\" (UID: \"20956a1e-47bb-4e89-9731-166265101190\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:43.026156 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:43.026060    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:43.328995 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:43.328967    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"]
Sep 19 13:06:43.335088 ip-10-0-136-161 kubenswrapper[2568]: W0919 13:06:43.335048    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20956a1e_47bb_4e89_9731_166265101190.slice/crio-5fa85a51c325799bf001a532f8a1229bae5d429c02311cfa21daacaf13075155 WatchSource:0}: Error finding container 5fa85a51c325799bf001a532f8a1229bae5d429c02311cfa21daacaf13075155: Status 404 returned error can't find the container with id 5fa85a51c325799bf001a532f8a1229bae5d429c02311cfa21daacaf13075155
Sep 19 13:06:43.336608 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:43.336584    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 13:06:44.288879 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:44.288843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf" event={"ID":"20956a1e-47bb-4e89-9731-166265101190","Type":"ContainerStarted","Data":"5fa85a51c325799bf001a532f8a1229bae5d429c02311cfa21daacaf13075155"}
Sep 19 13:06:46.298795 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:46.298756    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf" event={"ID":"20956a1e-47bb-4e89-9731-166265101190","Type":"ContainerStarted","Data":"7b18da73862a2afed9454639dfe23565594dadc61bbd478e54d2184a8ec94bbe"}
Sep 19 13:06:46.299262 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:46.298828    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:06:46.318615 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:46.318568    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf" podStartSLOduration=1.914014807 podStartE2EDuration="4.318553363s" totalImagesPullingTime="2.404538556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 13:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:06:46.316964822 +0000 UTC m=+2490.606520593" watchObservedRunningTime="2026-09-19 13:06:46.318553363 +0000 UTC m=+2490.608109132"
Sep 19 13:06:52.309064 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:06:52.309032    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-7gfwf"
Sep 19 13:07:11.490274 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:11.490246    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-fsxcs_6aee7517-57b6-474e-af96-b99b73000121/copy-manifests/0.log"
Sep 19 13:07:11.502955 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:11.502929    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-fsxcs_6aee7517-57b6-474e-af96-b99b73000121/manager/0.log"
Sep 19 13:07:11.957173 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:11.957122    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68c86dfd56-7gfwf_20956a1e-47bb-4e89-9731-166265101190/maas-api/0.log"
Sep 19 13:07:12.179190 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:12.179165    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830365-ndpqw_911cff9b-7621-4354-99ee-71e898cf49df/cleanup/0.log"
Sep 19 13:07:14.203778 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:14.203735    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt_b6f0e431-f923-40a1-b0d3-6466b2621066/istio-proxy/0.log"
Sep 19 13:07:14.326432 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:14.326401    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fr2wz_dc824c90-83a8-49e1-a2c6-80ae32dae915/discovery/0.log"
Sep 19 13:07:14.660996 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:14.660965    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/1.log"
Sep 19 13:07:14.893701 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:14.893672    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 13:07:15.544271 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:15.544237    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-nk4qw_11bd19f3-fc86-4eee-a138-72338a4417ca/main/0.log"
Sep 19 13:07:15.768835 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:15.768797    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8hr2l_05ba6e4b-51ca-4bed-8f1c-956a41e83c18/main/0.log"
Sep 19 13:07:25.224861 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:25.224825    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-658s9_6d9c3687-5584-4740-8d07-8cc36bb8213b/global-pull-secret-syncer/0.log"
Sep 19 13:07:25.413243 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:25.413214    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sbsbv_aee997a9-8dd4-4ff1-897f-1850eec78e94/konnectivity-agent/0.log"
Sep 19 13:07:25.531264 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:25.531193    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-161.ec2.internal_657c4b0f272c5c6a99d291261947b4fd/haproxy/0.log"
Sep 19 13:07:32.096094 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.096052    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-77nr2_96a74b53-50b6-45ce-8568-3af5f7d227a8/monitoring-plugin/0.log"
Sep 19 13:07:32.216292 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.216262    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sl6r_103274cb-e53d-4cc1-b242-036ae4a72dfb/node-exporter/0.log"
Sep 19 13:07:32.238272 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.238189    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sl6r_103274cb-e53d-4cc1-b242-036ae4a72dfb/kube-rbac-proxy/0.log"
Sep 19 13:07:32.260350 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.260332    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sl6r_103274cb-e53d-4cc1-b242-036ae4a72dfb/init-textfile/0.log"
Sep 19 13:07:32.373445 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.373415    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pghsz_265f9c7b-2277-4810-a78f-6ac4900c0ec1/kube-rbac-proxy-main/0.log"
Sep 19 13:07:32.400370 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.400348    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pghsz_265f9c7b-2277-4810-a78f-6ac4900c0ec1/kube-rbac-proxy-self/0.log"
Sep 19 13:07:32.425780 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.425750    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pghsz_265f9c7b-2277-4810-a78f-6ac4900c0ec1/openshift-state-metrics/0.log"
Sep 19 13:07:32.486374 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.486353    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/prometheus/0.log"
Sep 19 13:07:32.516744 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.516659    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/config-reloader/0.log"
Sep 19 13:07:32.536186 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.536165    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/thanos-sidecar/0.log"
Sep 19 13:07:32.559389 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.559363    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/kube-rbac-proxy-web/0.log"
Sep 19 13:07:32.582188 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.582168    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/kube-rbac-proxy/0.log"
Sep 19 13:07:32.604119 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.604099    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/kube-rbac-proxy-thanos/0.log"
Sep 19 13:07:32.626133 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:32.626111    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7527832b-2593-448c-a6eb-01cb401d7105/init-config-reloader/0.log"
Sep 19 13:07:35.390018 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.389981    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"]
Sep 19 13:07:35.390394 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.390306    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:07:35.390394 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.390317    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="02ae1836-8a3a-46ff-8f37-6bf2ad5182dd" containerName="istio-proxy"
Sep 19 13:07:35.390394 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.390376    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b361b4d4-bb4a-47e5-83a5-bd64cf0f6390" containerName="istio-proxy"
Sep 19 13:07:35.393701 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.393683    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.396134 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.396110    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrtv4\"/\"openshift-service-ca.crt\""
Sep 19 13:07:35.396983 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.396967    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrtv4\"/\"kube-root-ca.crt\""
Sep 19 13:07:35.397046 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.397019    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rrtv4\"/\"default-dockercfg-xbfxk\""
Sep 19 13:07:35.400210 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.400184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"]
Sep 19 13:07:35.477559 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.477527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-podres\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.477728 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.477568    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-sys\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.477728 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.477592    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-lib-modules\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.477728 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.477613    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-proc\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.477728 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.477664    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gd6q\" (UniqueName: \"kubernetes.io/projected/189b6b20-9c82-465d-927f-c08b7b555cbf-kube-api-access-9gd6q\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578727 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578692    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gd6q\" (UniqueName: \"kubernetes.io/projected/189b6b20-9c82-465d-927f-c08b7b555cbf-kube-api-access-9gd6q\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578924 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-podres\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578924 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578785    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-sys\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578924 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-lib-modules\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578924 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-proc\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.578924 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578905    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-sys\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.579125 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578962    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-podres\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.579125 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578980    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-proc\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.579125 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.578962    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/189b6b20-9c82-465d-927f-c08b7b555cbf-lib-modules\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.586729 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.586707    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gd6q\" (UniqueName: \"kubernetes.io/projected/189b6b20-9c82-465d-927f-c08b7b555cbf-kube-api-access-9gd6q\") pod \"perf-node-gather-daemonset-67f4p\" (UID: \"189b6b20-9c82-465d-927f-c08b7b555cbf\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.631534 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.631511    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-gthsf_e76d3cc9-ef1c-4058-a1fd-7f8684a9792f/download-server/0.log"
Sep 19 13:07:35.705230 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.705148    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:35.976875 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:35.976848    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"]
Sep 19 13:07:35.979491 ip-10-0-136-161 kubenswrapper[2568]: W0919 13:07:35.979467    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod189b6b20_9c82_465d_927f_c08b7b555cbf.slice/crio-daf6ff10b01faf4111757cc22e4f1cd4951c0f5d519dd0d9f4ad76f4f0f510f4 WatchSource:0}: Error finding container daf6ff10b01faf4111757cc22e4f1cd4951c0f5d519dd0d9f4ad76f4f0f510f4: Status 404 returned error can't find the container with id daf6ff10b01faf4111757cc22e4f1cd4951c0f5d519dd0d9f4ad76f4f0f510f4
Sep 19 13:07:36.524415 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:36.524377    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p" event={"ID":"189b6b20-9c82-465d-927f-c08b7b555cbf","Type":"ContainerStarted","Data":"79490c1789930bd42fac37b56e5b1bf271916da2fda7439b04c2b6c3aefac6d7"}
Sep 19 13:07:36.524899 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:36.524434    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:36.524899 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:36.524456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p" event={"ID":"189b6b20-9c82-465d-927f-c08b7b555cbf","Type":"ContainerStarted","Data":"daf6ff10b01faf4111757cc22e4f1cd4951c0f5d519dd0d9f4ad76f4f0f510f4"}
Sep 19 13:07:36.541237 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:36.541191    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p" podStartSLOduration=1.541176304 podStartE2EDuration="1.541176304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 13:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:07:36.540178689 +0000 UTC m=+2540.829734459" watchObservedRunningTime="2026-09-19 13:07:36.541176304 +0000 UTC m=+2540.830732074"
Sep 19 13:07:37.568202 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:37.568161    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vgq6p_a2c8aff9-8f5e-4542-a402-547ae16192a6/dns/0.log"
Sep 19 13:07:37.593682 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:37.593650    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vgq6p_a2c8aff9-8f5e-4542-a402-547ae16192a6/kube-rbac-proxy/0.log"
Sep 19 13:07:37.657074 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:37.657043    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8v6pn_b15a0a7b-c722-403f-a655-48d1961671c1/dns-node-resolver/0.log"
Sep 19 13:07:38.214235 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:38.214194    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b79bc9699-l2nzx_12a7200d-b7c5-4b65-9541-1091a186fb31/registry/0.log"
Sep 19 13:07:38.284826 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:38.284799    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vnjc7_bd8fbf76-c9ce-4e7c-9629-5bc0c819bad5/node-ca/0.log"
Sep 19 13:07:39.183056 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:39.183023    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt_b6f0e431-f923-40a1-b0d3-6466b2621066/istio-proxy/0.log"
Sep 19 13:07:40.035996 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:40.035947    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fr2wz_dc824c90-83a8-49e1-a2c6-80ae32dae915/discovery/0.log"
Sep 19 13:07:40.109774 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:40.109744    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/0.log"
Sep 19 13:07:40.115520 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:40.115475    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/1.log"
Sep 19 13:07:40.192217 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:40.192183    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 13:07:41.127563 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:41.127530    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fkcc7_a3d03cb6-902e-40e1-8a19-3241c94325ba/serve-healthcheck-canary/0.log"
Sep 19 13:07:41.631395 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:41.631365    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849f4_e436cfc9-8f66-4fb2-a463-4aa69d714052/kube-rbac-proxy/0.log"
Sep 19 13:07:41.651105 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:41.651079    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849f4_e436cfc9-8f66-4fb2-a463-4aa69d714052/exporter/0.log"
Sep 19 13:07:41.674933 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:41.674908    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849f4_e436cfc9-8f66-4fb2-a463-4aa69d714052/extractor/0.log"
Sep 19 13:07:42.538968 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:42.538942    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-67f4p"
Sep 19 13:07:44.749233 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:44.749178    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-fsxcs_6aee7517-57b6-474e-af96-b99b73000121/manager/0.log"
Sep 19 13:07:44.769092 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:44.769063    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-fsxcs_6aee7517-57b6-474e-af96-b99b73000121/copy-manifests/0.log"
Sep 19 13:07:45.773271 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:45.773242    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt_b6f0e431-f923-40a1-b0d3-6466b2621066/istio-proxy/0.log"
Sep 19 13:07:46.234955 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:46.234928    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6bdf74d45f-4x9p7_e8b39b0c-e918-4b29-805c-12bdbb78dd51/manager/0.log"
Sep 19 13:07:46.600047 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:46.600014    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fr2wz_dc824c90-83a8-49e1-a2c6-80ae32dae915/discovery/0.log"
Sep 19 13:07:46.658208 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:46.658173    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/0.log"
Sep 19 13:07:46.663463 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:46.663437    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/1.log"
Sep 19 13:07:46.727940 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:46.727907    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 13:07:48.220335 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:48.220301    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-44sbt_b6f0e431-f923-40a1-b0d3-6466b2621066/istio-proxy/0.log"
Sep 19 13:07:49.098126 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:49.098094    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fr2wz_dc824c90-83a8-49e1-a2c6-80ae32dae915/discovery/0.log"
Sep 19 13:07:49.170022 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:49.169990    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/0.log"
Sep 19 13:07:49.175612 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:49.175584    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-sc2x2_fbb6d78f-90df-4ed6-b633-7735ba4e54fb/istio-proxy/1.log"
Sep 19 13:07:49.240833 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:49.240808    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9qmjq_980f4550-872a-4378-bd11-16c871a3b369/payload-processing/0.log"
Sep 19 13:07:53.440729 ip-10-0-136-161 kubenswrapper[2568]: I0919 13:07:53.440697    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qt8cr_049d77b8-c91f-48ce-badc-f9b904873b7f/kube-multus-additional-cni-plugins/0.log"