Sep 15 20:56:01.717550 ip-10-0-131-2 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 15 20:56:01.717560 ip-10-0-131-2 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 15 20:56:01.717567 ip-10-0-131-2 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 15 20:56:01.717784 ip-10-0-131-2 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 15 20:56:11.843727 ip-10-0-131-2 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 15 20:56:11.843751 ip-10-0-131-2 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 61d3a390c2de4567a4d92e17f37998ea --
Sep 15 20:58:11.078037 ip-10-0-131-2 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 20:58:11.431285 ip-10-0-131-2 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:58:11.431285 ip-10-0-131-2 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 20:58:11.431285 ip-10-0-131-2 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:58:11.431285 ip-10-0-131-2 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 20:58:11.431285 ip-10-0-131-2 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:58:11.431959 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.431377    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 20:58:11.434989 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.434973    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.434989 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.434988    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.434992    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.434995    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.434998    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435000    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435004    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435007    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435009    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435012    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435014    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435017    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435020    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435022    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435025    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435029    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435032    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435035    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435037    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435040    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435042    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.435056 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435044    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435047    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435049    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435052    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435054    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435057    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435062    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435067    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435070    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435073    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435076    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435079    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435083    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435086    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435088    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435090    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435093    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435096    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435098    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435100    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.435870 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435103    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435105    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435108    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435111    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435113    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435116    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435118    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435121    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435124    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435126    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435130    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435134    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435137    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435140    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435399    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435410    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435415    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435420    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435425    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.436522 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435430    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435435    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435439    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435443    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435448    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435453    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435458    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435462    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435472    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435477    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435482    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435485    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435496    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435502    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435509    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435514    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435518    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435523    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435527    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.437058 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435536    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435541    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435555    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435560    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435565    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435569    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435574    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435578    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435584    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435589    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435593    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435597    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435602    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435611    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435616    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435620    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435624    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435628    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435632    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.435637    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.437535 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.436991    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437001    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437005    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437008    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437011    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437014    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437017    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437019    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437022    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437026    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437028    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437031    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437034    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437037    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437039    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437043    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437046    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437048    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437050    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437053    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.438057 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437055    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437058    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437061    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437063    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437066    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437069    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437072    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437074    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437076    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437079    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437081    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437084    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437087    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437090    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437093    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437095    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437098    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437100    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437103    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437105    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.438568 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437108    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437110    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437113    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437117    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437121    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437123    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437127    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437129    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437131    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437135    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437138    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437140    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437143    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437145    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437148    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437150    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437153    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437156    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437158    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437161    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.439105 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437163    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437165    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437168    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437171    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437174    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437179    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437182    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437184    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437187    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437189    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437192    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437195    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437199    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437201    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437204    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437206    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437209    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437212    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437214    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.439592 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437217    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437221    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437224    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437227    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437230    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437233    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437236    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437239    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437242    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437244    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437246    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437249    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437251    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437254    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437256    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437260    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437262    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437265    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437270    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.440083 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437275    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437658    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437663    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437666    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437669    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437671    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437675    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437677    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437680    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437682    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437685    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437688    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437690    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437693    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437696    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437698    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437701    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437703    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437706    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437709    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.440565 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437711    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437714    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437716    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437719    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437721    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437723    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437726    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437750    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437753    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437757    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437760    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437764    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437767    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437770    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437772    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437775    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437777    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437780    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437784    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437789    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.441090 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437791    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437794    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437796    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437799    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437803    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437806    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437809    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437811    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437814    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437816    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437819    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437821    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437824    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437827    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437829    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437832    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437835    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437837    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437840    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.441603 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437843    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437845    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437847    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437850    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437854    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437856    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437859    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437861    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437864    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437866    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437868    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437872    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437875    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437879    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437882    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437885    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437887    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437891    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437894    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.442186 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437897    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437899    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437902    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437904    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437907    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437909    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437912    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437915    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437917    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437920    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437922    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437925    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437927    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437930    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437932    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437934    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437937    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437941    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437943    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437946    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.442651 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437948    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.437951    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438017    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438057    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438063    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438067    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438071    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438077    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438081    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438085    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438088    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438092    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438095    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438098    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438101    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438104    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438107    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438110    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438112    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438119    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438123    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438126    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438129    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438132    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 20:58:11.443161 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438135    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438139    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438142    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438145    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438148    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438152    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438155    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438158    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438161    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438163    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438167    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438170    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438173    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438176    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438179    2577 flags.go:64] FLAG: --enable-server="true"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438182    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438186    2577 flags.go:64] FLAG: --event-burst="100"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438189    2577 flags.go:64] FLAG: --event-qps="50"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438192    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438194    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438197    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438201    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438203    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438206    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438209    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 15 20:58:11.443775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438212    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438215    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438219    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438222    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438225    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438228    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438231    2577 flags.go:64] FLAG: --feature-gates=""
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438234    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438237    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438240    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438243    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438246    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438250    2577 flags.go:64] FLAG: --help="false"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438253    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438256    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438259    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438262    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438265    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438268    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438271    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438273    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438276    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438279    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 20:58:11.444399 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438282    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438285    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438288    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438291    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438294    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438297    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438299    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438302    2577 flags.go:64] FLAG: --lock-file=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438305    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438308    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438310    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438317    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438320    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438323    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438325    2577 flags.go:64] FLAG: --logging-format="text"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438329    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438331    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438334    2577 flags.go:64] FLAG: --manifest-url=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438337    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438346    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438349    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438354    2577 flags.go:64] FLAG: --max-pods="110"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438357    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438360    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438363    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 20:58:11.445012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438366    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438369    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438372    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438375    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438383    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438386    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438388    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438391    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438394    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438401    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438403    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438407    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438409    2577 flags.go:64] FLAG: --port="10250"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438412    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438415    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-058e1c38c1a8bc12d"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438418    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438421    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438424    2577 flags.go:64] FLAG: --register-node="true"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438426    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438435    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438438    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438440    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438443    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438446    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438450    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 20:58:11.445675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438452    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438455    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438458    2577 flags.go:64] FLAG: --runonce="false"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438463    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438466    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438469    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438471    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438474    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438477    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438480    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438483    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438486    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438489    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438492    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438494    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438497    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438501    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438507    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438512    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438515    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438517    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438524    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438527    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438530    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438532    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 20:58:11.446388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438535    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438539    2577 flags.go:64] FLAG: --v="2"
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438543    2577 flags.go:64] FLAG: --version="false"
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438546    2577 flags.go:64] FLAG: --vmodule=""
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438550    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.438553    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438679    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438684    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438688    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438690    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438694    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438697    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438699    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438702    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438704    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438707    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438709    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438712    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438714    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438718    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.447033 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438722    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438725    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438728    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438730    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438733    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438755    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438758    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438761    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438763    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438766    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438770    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438774    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438777    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438782    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438784    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438787    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438790    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438793    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438795    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438798    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.447534 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438800    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438803    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438806    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438808    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438811    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438814    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438816    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438819    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438821    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438824    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438826    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438829    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438832    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438834    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438837    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438840    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438842    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438845    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438848    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438851    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.448050 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438853    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438856    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438859    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438861    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438864    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438868    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438870    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438873    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438876    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438878    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438881    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438884    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438886    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438889    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438891    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438894    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438896    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438899    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438901    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438904    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.448550 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438908    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438911    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438914    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438916    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438919    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438921    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438924    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438926    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438929    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438931    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438933    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438936    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438939    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438941    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438944    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438947    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438949    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438952    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438955    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438958    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.449082 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438960    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438963    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438966    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438968    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.438970    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.439666    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.447479    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.447494    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.448043    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.448050    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448110    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448115    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448118    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448121    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.449621 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448124    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448127    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448129    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448132    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448135    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448137    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448140    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448142    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448145    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448150    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448154    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448157    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448160    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448163    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448166    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448168    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448171    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448173    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448176    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448179    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.450012 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448181    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448184    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448187    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448189    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448193    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448197    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448199    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448202    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448205    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448207    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448210    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448212    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448215    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448218    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448221    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448223    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448227    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448230    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448233    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.450514 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448236    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448239    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448241    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448244    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448247    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448250    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448253    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448256    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448259    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448261    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448264    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448266    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448269    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448272    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448275    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448277    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448280    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448283    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448285    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448288    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.451018 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448291    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448293    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448296    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448298    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448301    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448303    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448306    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448308    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448311    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448313    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448316    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448319    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448321    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448324    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448326    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448329    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448332    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448334    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448337    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448339    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.451554 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448341    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448344    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448346    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448349    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448351    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448354    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448356    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448359    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448361    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448364    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448367    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448370    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448372    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448375    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448377    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.452055 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448379    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.448385    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448498    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448502    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448505    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448508    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448510    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448513    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448517    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448520    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448523    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448526    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448528    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448531    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448534    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:58:11.452445 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448537    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448539    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448542    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448544    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448546    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448549    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448551    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448554    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448557    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448560    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448562    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448565    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448568    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448570    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448573    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448575    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448578    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448580    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448582    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448585    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448587    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:58:11.452867 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448590    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448592    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448594    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448597    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448599    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448602    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448604    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448607    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448610    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448612    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448615    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448618    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448620    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448623    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448625    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448628    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448630    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448632    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448635    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:58:11.453394 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448638    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448642    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448645    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448648    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448651    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448654    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448657    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448659    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448662    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448664    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448667    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448669    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448672    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448674    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448677    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448679    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448682    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448684    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448687    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:58:11.453977 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448689    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448692    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448695    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448697    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448699    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448702    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448705    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448707    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448710    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448712    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448715    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448717    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448720    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448723    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448726    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448729    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448733    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448753    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448758    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:58:11.454448 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448762    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448766    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448770    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448773    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448776    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448779    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448781    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:11.448784    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.448789    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.448887    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.451172    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.452098    2577 server.go:1019] "Starting client certificate rotation"
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.452205    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:58:11.454934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.452240    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:58:11.455290 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.454499    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:58:11.455778 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.455760    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:58:11.466922 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.466902    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 15 20:58:11.470294 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.470277    2577 log.go:25] "Validated CRI v1 image API"
Sep 15 20:58:11.471191 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.471175    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 20:58:11.474812 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.474786    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7deb70a2-4e40-47ab-b09c-efaf3e34f676:/dev/nvme0n1p3 d6c07e71-7361-4944-ac1b-69cb0d3d02d7:/dev/nvme0n1p4]
Sep 15 20:58:11.474870 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.474811    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 20:58:11.478518 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.478501    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:58:11.480267 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.480159    2577 manager.go:216] Machine: {Timestamp:2026-09-15 20:58:11.478944581 +0000 UTC m=+0.296137835 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115046 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c6e42862cf9ef789a2e677eab98d8 SystemUUID:ec2c6e42-862c-f9ef-789a-2e677eab98d8 BootID:61d3a390-c2de-4567-a4d9-2e17f37998ea Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:23:6c:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:23:6c:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:0d:21:6b:eb:96 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 20:58:11.480267 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.480262    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 20:58:11.480383 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.480339    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 20:58:11.482846 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.482826    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 20:58:11.482977 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.482849    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-2.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 20:58:11.483564 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.483555    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 20:58:11.483595 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.483567    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 20:58:11.483595 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.483574    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:58:11.484418 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.484408    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:58:11.485524 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.485514    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 20:58:11.488621 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488610    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:58:11.488750 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488730    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 20:58:11.488782 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488771    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 20:58:11.488782 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488781    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 20:58:11.488840 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488798    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 20:58:11.488840 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488808    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 15 20:58:11.488840 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.488817    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 20:58:11.489165 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.489153    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:58:11.489205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.489172    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:58:11.491075 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.491058    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 20:58:11.493851 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.493835    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 20:58:11.493923 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.493866    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 20:58:11.495958 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495946    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495965    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495971    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495976    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495983    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.495989    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 20:58:11.496002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496002    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 20:58:11.496162 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496007    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 20:58:11.496162 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496013    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 20:58:11.496162 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496020    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 20:58:11.496162 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496034    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 20:58:11.496162 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496039    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 20:58:11.496779 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496767    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 20:58:11.496857 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.496782    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 20:58:11.497722 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.497680    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-2.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 20:58:11.497807 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.497679    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 20:58:11.498851 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.498835    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-2.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 20:58:11.500528 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.500515    2577 server.go:1268] "Started kubelet"
Sep 15 20:58:11.500625 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.500604    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 20:58:11.500704 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.500657    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 20:58:11.500770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.500734    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 20:58:11.501381 ip-10-0-131-2 systemd[1]: Started Kubernetes Kubelet.
Sep 15 20:58:11.502471 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.502201    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 20:58:11.503127 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.503106    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 20:58:11.504922 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.504897    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x7727"
Sep 15 20:58:11.506009 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.505971    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 20:58:11.508278 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.508262    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 20:58:11.508362 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.508280    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 20:58:11.508362 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.508352    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.508930 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.508865    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 20:58:11.508930 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.508874    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 20:58:11.508930 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.508894    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 20:58:11.509133 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.509025    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 20:58:11.509133 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.509035    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 20:58:11.509256 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.509235    2577 factory.go:55] Registering systemd factory
Sep 15 20:58:11.509256 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.509254    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 15 20:58:11.509468 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.508636    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-2.ec2.internal.18d59a208a5aded1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-2.ec2.internal,UID:ip-10-0-131-2.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-2.ec2.internal,},FirstTimestamp:2026-09-15 20:58:11.500482257 +0000 UTC m=+0.317675507,LastTimestamp:2026-09-15 20:58:11.500482257 +0000 UTC m=+0.317675507,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-2.ec2.internal,}"
Sep 15 20:58:11.514003 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.513918    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x7727"
Sep 15 20:58:11.514003 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.513943    2577 factory.go:153] Registering CRI-O factory
Sep 15 20:58:11.514003 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.513959    2577 factory.go:223] Registration of the crio container factory successfully
Sep 15 20:58:11.514136 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.514009    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 20:58:11.514136 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.514028    2577 factory.go:103] Registering Raw factory
Sep 15 20:58:11.514136 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.514044    2577 manager.go:1196] Started watching for new ooms in manager
Sep 15 20:58:11.514636 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.514620    2577 manager.go:349] Starting recovery of all containers
Sep 15 20:58:11.517331 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.517304    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-2.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 20:58:11.517432 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.517412    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 20:58:11.523441 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.523426    2577 manager.go:354] Recovery completed
Sep 15 20:58:11.527403 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.527389    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.530020 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.530004    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.530084 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.530035    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.530084 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.530048    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.531012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531000    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 20:58:11.531012 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531011    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 20:58:11.531092 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531028    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:58:11.531092 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531032    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.531775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531757    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.531840 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531789    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.531840 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.531803    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.532063 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.532050    2577 policy_none.go:49] "None policy: Start"
Sep 15 20:58:11.532125 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.532069    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 20:58:11.532125 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.532082    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 20:58:11.532984 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.532973    2577 policy_none.go:47] "Start"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.571210    2577 manager.go:337] "Starting Device Plugin manager"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.571255    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.571269    2577 server.go:87] "Starting device plugin registration server"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.571608    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.571622    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.572595    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.572848    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.572860    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.573009    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 20:58:11.580781 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.573325    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.651484 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.651452    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 20:58:11.652826 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.652810    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 20:58:11.652931 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.652861    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 20:58:11.652931 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.652887    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 20:58:11.653030 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.652930    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 20:58:11.655275 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.655257    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:58:11.672733 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.672710    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.673535 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.673510    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.673609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.673543    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.673609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.673554    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.673609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.673575    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.681037 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.680994    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.681084 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.681045    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-2.ec2.internal\": node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.696480 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.696464    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.753816 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.753792    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"]
Sep 15 20:58:11.753918 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.753855    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.756803 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.756789    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.756859 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.756816    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.756859 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.756827    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.757990 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.757978    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.758099 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.758136 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758115    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.758846 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758822    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.758907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758850    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.758907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758865    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.758907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758879    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.758907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758894    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.758907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.758906    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.760062 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.760048    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.760108 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.760079    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:58:11.760849 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.760834    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:58:11.760900 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.760856    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:58:11.760900 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.760865    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:58:11.790289 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.790267    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-2.ec2.internal\" not found" node="ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.794537 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.794524    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-2.ec2.internal\" not found" node="ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.796521 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.796507    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.810907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.810890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.810982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.810915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.810982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.810931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ed654b3d866e9d8cd5563c046da053f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-2.ec2.internal\" (UID: \"4ed654b3d866e9d8cd5563c046da053f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.897517 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.897477    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:11.911891 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.911870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.911891 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.911888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.912043 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.911915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.912043 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.911940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ed654b3d866e9d8cd5563c046da053f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-2.ec2.internal\" (UID: \"4ed654b3d866e9d8cd5563c046da053f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.912043 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.911993    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ed654b3d866e9d8cd5563c046da053f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-2.ec2.internal\" (UID: \"4ed654b3d866e9d8cd5563c046da053f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.912043 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:11.912002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c25b0db8bec3d49e89def24b24b07505-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal\" (UID: \"c25b0db8bec3d49e89def24b24b07505\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:11.998314 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:11.998252    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.093782 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.093724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:12.098496 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.098237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:12.098496 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.098335    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.198821 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.198779    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.299328 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.299254    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.399735 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.399706    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.452277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.452251    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 20:58:12.452648 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.452399    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 20:58:12.499809 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.499775    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.516879 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.516837    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 20:53:12 +0000 UTC" deadline="2028-04-08 05:37:14.533868348 +0000 UTC"
Sep 15 20:58:12.516879 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.516875    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13688h39m2.016995962s"
Sep 15 20:58:12.589223 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:12.589195    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed654b3d866e9d8cd5563c046da053f.slice/crio-9f8794ce97f3889114a0501555a342097f2e98f2cd62dc8695ef44c1c98da740 WatchSource:0}: Error finding container 9f8794ce97f3889114a0501555a342097f2e98f2cd62dc8695ef44c1c98da740: Status 404 returned error can't find the container with id 9f8794ce97f3889114a0501555a342097f2e98f2cd62dc8695ef44c1c98da740
Sep 15 20:58:12.593310 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.593291    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:58:12.600530 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.600514    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.626284 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:12.626259    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25b0db8bec3d49e89def24b24b07505.slice/crio-e4a84ff1262170aea2d3671e2898c5111b3f08f75c9f654d7bad718bbc37417b WatchSource:0}: Error finding container e4a84ff1262170aea2d3671e2898c5111b3f08f75c9f654d7bad718bbc37417b: Status 404 returned error can't find the container with id e4a84ff1262170aea2d3671e2898c5111b3f08f75c9f654d7bad718bbc37417b
Sep 15 20:58:12.656641 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.656590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal" event={"ID":"4ed654b3d866e9d8cd5563c046da053f","Type":"ContainerStarted","Data":"9f8794ce97f3889114a0501555a342097f2e98f2cd62dc8695ef44c1c98da740"}
Sep 15 20:58:12.657670 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.657651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal" event={"ID":"c25b0db8bec3d49e89def24b24b07505","Type":"ContainerStarted","Data":"e4a84ff1262170aea2d3671e2898c5111b3f08f75c9f654d7bad718bbc37417b"}
Sep 15 20:58:12.669164 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.669145    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:58:12.701258 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.701239    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.792055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:12.792024    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:58:12.802055 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.802029    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:12.902577 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:12.902504    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:13.002613 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:13.002569    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-2.ec2.internal\" not found"
Sep 15 20:58:13.004786 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:13.004765    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:58:13.009155 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:13.009120    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:13.018542 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:13.018520    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:58:13.019569 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:13.019543    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal"
Sep 15 20:58:13.028701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:13.028679    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:58:14.521112 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.521032    2577 apiserver.go:52] "Watching apiserver"
Sep 15 20:58:14.528754 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.528732    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 20:58:14.531334 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.531302    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal","openshift-multus/multus-additional-cni-plugins-h57dk","openshift-multus/network-metrics-daemon-s8465","openshift-network-diagnostics/network-check-target-9hj9w","openshift-network-operator/iptables-alerter-98b7q","openshift-ovn-kubernetes/ovnkube-node-78t57","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5","openshift-cluster-node-tuning-operator/tuned-977rj","openshift-dns/node-resolver-s7489","openshift-image-registry/node-ca-c684b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal","openshift-multus/multus-tl79d","kube-system/global-pull-secret-syncer-p4qc5","kube-system/konnectivity-agent-zp94c"]
Sep 15 20:58:14.532848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.532829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.533934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.533912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.534024 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.533987    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:14.535206 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.535181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.535296 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.535231    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:14.535535 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.535514    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z5b9k\""
Sep 15 20:58:14.535647 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.535517    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 20:58:14.535705 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.535521    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.535705 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.535521    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.536344 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.536297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:14.536426 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.536358    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:14.539320 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.539266    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 20:58:14.540282 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.540266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.543123 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.542982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.543290 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.543000    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7xrrg\""
Sep 15 20:58:14.543290 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.543053    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.543420 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.543079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 20:58:14.543484 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.543469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.544030 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.544013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.545323 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.545287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.546004 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.545986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 20:58:14.546390 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546372    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.546774 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546511    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.546774 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546623    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.546774 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 20:58:14.546938 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8cgfm\""
Sep 15 20:58:14.546938 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.546899    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 20:58:14.547068 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.547049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.547780 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.547762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 20:58:14.547860 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.547808    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.548215 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.548018    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pxpd8\""
Sep 15 20:58:14.548215 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.548088    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pnmqq\""
Sep 15 20:58:14.548215 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.548139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 20:58:14.548411 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.548232    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 20:58:14.549089 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.548831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.549285 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.549266    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 20:58:14.549549 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.549533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sct9r\""
Sep 15 20:58:14.549799 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.549786    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.549865 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.549787    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 20:58:14.549865 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.549816    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 20:58:14.550180 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.550098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.550609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.550437    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:58:14.550701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.550682    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.551445 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.551363    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.551851 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.551665    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 20:58:14.551851 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.551717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.552086 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.552594 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552370    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zxzzq\""
Sep 15 20:58:14.552845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552696    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:14.552845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-registration-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.552845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-dbus\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.552845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f883507-b4c4-40b9-992e-c0bcb81cc20e-iptables-alerter-script\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn9cq\" (UniqueName: \"kubernetes.io/projected/6f883507-b4c4-40b9-992e-c0bcb81cc20e-kube-api-access-nn9cq\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552839    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-socket-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-sys-fs\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6ss4\" (UniqueName: \"kubernetes.io/projected/0f95378e-7403-4add-a255-67713c570ade-kube-api-access-q6ss4\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552968    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.552991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-device-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.553101 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-kubelet-config\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.553674 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.553674 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl5zp\" (UniqueName: \"kubernetes.io/projected/9a072b95-451a-4006-8b19-5ecf3ef181b0-kube-api-access-gl5zp\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.553674 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.553228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f883507-b4c4-40b9-992e-c0bcb81cc20e-host-slash\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.554444 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.554007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4ng2n\""
Sep 15 20:58:14.554507 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.554449    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 20:58:14.555445 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.555009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4x6zs\""
Sep 15 20:58:14.572352 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.572329    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5f8s2"
Sep 15 20:58:14.577093 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.577072    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5f8s2"
Sep 15 20:58:14.640388 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.640361    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 20:58:14.654283 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f883507-b4c4-40b9-992e-c0bcb81cc20e-host-slash\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.654394 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-ovn\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.654394 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.654394 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-cnibin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654394 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f883507-b4c4-40b9-992e-c0bcb81cc20e-host-slash\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-systemd-units\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-netns\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-socket-dir-parent\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-multus\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-hostroot\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-conf\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6ss4\" (UniqueName: \"kubernetes.io/projected/0f95378e-7403-4add-a255-67713c570ade-kube-api-access-q6ss4\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-systemd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-multus-certs\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp8h7\" (UniqueName: \"kubernetes.io/projected/15a19d1c-f032-4458-be72-a297de17625e-kube-api-access-mp8h7\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.654848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e67c6e70-a7b4-443e-b069-892373a430a1-hosts-file\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-systemd\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-tuned\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl5zp\" (UniqueName: \"kubernetes.io/projected/9a072b95-451a-4006-8b19-5ecf3ef181b0-kube-api-access-gl5zp\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-netd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovn-node-metrics-cert\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.654976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84h9l\" (UniqueName: \"kubernetes.io/projected/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-kube-api-access-84h9l\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-conf-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-cni-binary-copy\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-bin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdr2h\" (UniqueName: \"kubernetes.io/projected/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-kube-api-access-gdr2h\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-registration-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.655205 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn9cq\" (UniqueName: \"kubernetes.io/projected/6f883507-b4c4-40b9-992e-c0bcb81cc20e-kube-api-access-nn9cq\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-registration-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-var-lib-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd315a5f-06a8-472b-8269-d87f8d05b72d-konnectivity-ca\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-tmp\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-socket-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-system-cni-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tlkk\" (UniqueName: \"kubernetes.io/projected/8ed722cd-2171-44aa-ac37-36725ac5c779-kube-api-access-2tlkk\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-lib-modules\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-kubelet-config\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-log-socket\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2dd47b8-6d65-4982-9541-0e3293d59c85-serviceca\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-kubelet\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655640    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-socket-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-etc-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-script-lib\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.655760 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kp9x\" (UniqueName: \"kubernetes.io/projected/e67c6e70-a7b4-443e-b069-892373a430a1-kube-api-access-2kp9x\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-binary-copy\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-kubelet\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-multus-daemon-config\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd315a5f-06a8-472b-8269-d87f8d05b72d-agent-certs\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-host\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-bin\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-config\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-sys\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-device-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-kubelet-config\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-slash\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-env-overrides\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-device-dir\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.656389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.655982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-node-log\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-etc-selinux\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-etc-kubernetes\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-cnibin\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2dd47b8-6d65-4982-9541-0e3293d59c85-host\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-modprobe-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-var-lib-kubelet\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-dbus\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f883507-b4c4-40b9-992e-c0bcb81cc20e-iptables-alerter-script\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-netns\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/723574f1-d85e-4b0e-9f37-9c0a96789c22-dbus\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67c6e70-a7b4-443e-b069-892373a430a1-tmp-dir\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-kubernetes\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-run\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657147 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-sys-fs\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-k8s-cni-cncf-io\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6dc7\" (UniqueName: \"kubernetes.io/projected/c2dd47b8-6d65-4982-9541-0e3293d59c85-kube-api-access-q6dc7\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f95378e-7403-4add-a255-67713c570ade-sys-fs\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.656731    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-system-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysconfig\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.656785    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.656806    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:15.15677044 +0000 UTC m=+2.943138426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.656853    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:15.156836713 +0000 UTC m=+2.943204706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.656869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.657715 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.657645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f883507-b4c4-40b9-992e-c0bcb81cc20e-iptables-alerter-script\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.660333 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.660308    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 20:58:14.662034 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.662013    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:14.662140 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.662037    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:14.662140 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.662056    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:14.662140 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:14.662110    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:15.162094993 +0000 UTC m=+2.948462998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:14.664304 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.664277    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6ss4\" (UniqueName: \"kubernetes.io/projected/0f95378e-7403-4add-a255-67713c570ade-kube-api-access-q6ss4\") pod \"aws-ebs-csi-driver-node-hm6p5\" (UID: \"0f95378e-7403-4add-a255-67713c570ade\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.664396 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.664284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn9cq\" (UniqueName: \"kubernetes.io/projected/6f883507-b4c4-40b9-992e-c0bcb81cc20e-kube-api-access-nn9cq\") pod \"iptables-alerter-98b7q\" (UID: \"6f883507-b4c4-40b9-992e-c0bcb81cc20e\") " pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.664396 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.664335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl5zp\" (UniqueName: \"kubernetes.io/projected/9a072b95-451a-4006-8b19-5ecf3ef181b0-kube-api-access-gl5zp\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:14.743751 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.743722    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:58:14.757893 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-modprobe-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-var-lib-kubelet\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-netns\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67c6e70-a7b4-443e-b069-892373a430a1-tmp-dir\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-kubernetes\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.757995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-run\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758023 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-netns\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-k8s-cni-cncf-io\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-var-lib-kubelet\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-modprobe-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6dc7\" (UniqueName: \"kubernetes.io/projected/c2dd47b8-6d65-4982-9541-0e3293d59c85-kube-api-access-q6dc7\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-system-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-run\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysconfig\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-kubernetes\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysconfig\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-ovn\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-system-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-cnibin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758215    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-k8s-cni-cncf-io\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-systemd-units\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-ovn\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.758382 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-netns\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-socket-dir-parent\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-multus\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-d\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-systemd-units\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67c6e70-a7b4-443e-b069-892373a430a1-tmp-dir\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-hostroot\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-multus\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-netns\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758370    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-hostroot\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-cnibin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-socket-dir-parent\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-conf\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-systemd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-multus-certs\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-sysctl-conf\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-systemd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-run-multus-certs\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp8h7\" (UniqueName: \"kubernetes.io/projected/15a19d1c-f032-4458-be72-a297de17625e-kube-api-access-mp8h7\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e67c6e70-a7b4-443e-b069-892373a430a1-hosts-file\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-systemd\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-tuned\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-netd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovn-node-metrics-cert\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84h9l\" (UniqueName: \"kubernetes.io/projected/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-kube-api-access-84h9l\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-conf-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-cni-binary-copy\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-bin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdr2h\" (UniqueName: \"kubernetes.io/projected/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-kube-api-access-gdr2h\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-var-lib-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd315a5f-06a8-472b-8269-d87f8d05b72d-konnectivity-ca\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.759680 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-tmp\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.758981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-system-cni-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tlkk\" (UniqueName: \"kubernetes.io/projected/8ed722cd-2171-44aa-ac37-36725ac5c779-kube-api-access-2tlkk\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-lib-modules\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-log-socket\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2dd47b8-6d65-4982-9541-0e3293d59c85-serviceca\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-kubelet\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-etc-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-script-lib\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kp9x\" (UniqueName: \"kubernetes.io/projected/e67c6e70-a7b4-443e-b069-892373a430a1-kube-api-access-2kp9x\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-binary-copy\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-kubelet\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-multus-daemon-config\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd315a5f-06a8-472b-8269-d87f8d05b72d-agent-certs\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-host\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-bin\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760301 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-config\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-sys\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-slash\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-env-overrides\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-conf-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-env-overrides\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-lib-modules\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.759962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-log-socket\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-cni-binary-copy\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd315a5f-06a8-472b-8269-d87f8d05b72d-konnectivity-ca\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760285    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2dd47b8-6d65-4982-9541-0e3293d59c85-serviceca\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-cni-bin\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-kubelet\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-etc-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-var-lib-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-system-cni-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-systemd\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.760983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-script-lib\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e67c6e70-a7b4-443e-b069-892373a430a1-hosts-file\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-netd\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-node-log\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.760993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-etc-kubernetes\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-cnibin\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2dd47b8-6d65-4982-9541-0e3293d59c85-host\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2dd47b8-6d65-4982-9541-0e3293d59c85-host\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-multus-cni-dir\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-etc-kubernetes\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-node-log\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-cnibin\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-run-ovn-kubernetes\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a19d1c-f032-4458-be72-a297de17625e-host-var-lib-kubelet\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ed722cd-2171-44aa-ac37-36725ac5c779-cni-binary-copy\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ed722cd-2171-44aa-ac37-36725ac5c779-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.761997 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-run-openvswitch\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-sys\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-cni-bin\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-host\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-host-slash\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15a19d1c-f032-4458-be72-a297de17625e-multus-daemon-config\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.761944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovnkube-config\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.763081 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.762906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-tmp\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.763496 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.763479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-ovn-node-metrics-cert\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.763705 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.763689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-etc-tuned\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.764064 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.764045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd315a5f-06a8-472b-8269-d87f8d05b72d-agent-certs\") pod \"konnectivity-agent-zp94c\" (UID: \"bd315a5f-06a8-472b-8269-d87f8d05b72d\") " pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.766712 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.766688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6dc7\" (UniqueName: \"kubernetes.io/projected/c2dd47b8-6d65-4982-9541-0e3293d59c85-kube-api-access-q6dc7\") pod \"node-ca-c684b\" (UID: \"c2dd47b8-6d65-4982-9541-0e3293d59c85\") " pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.767349 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.767329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tlkk\" (UniqueName: \"kubernetes.io/projected/8ed722cd-2171-44aa-ac37-36725ac5c779-kube-api-access-2tlkk\") pod \"multus-additional-cni-plugins-h57dk\" (UID: \"8ed722cd-2171-44aa-ac37-36725ac5c779\") " pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.767937 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.767921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84h9l\" (UniqueName: \"kubernetes.io/projected/4dc6e8d9-fc44-4d1a-b425-d24bcccb9277-kube-api-access-84h9l\") pod \"ovnkube-node-78t57\" (UID: \"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277\") " pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.768444 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.768425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdr2h\" (UniqueName: \"kubernetes.io/projected/26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7-kube-api-access-gdr2h\") pod \"tuned-977rj\" (UID: \"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7\") " pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.768910 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.768889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp8h7\" (UniqueName: \"kubernetes.io/projected/15a19d1c-f032-4458-be72-a297de17625e-kube-api-access-mp8h7\") pod \"multus-tl79d\" (UID: \"15a19d1c-f032-4458-be72-a297de17625e\") " pod="openshift-multus/multus-tl79d"
Sep 15 20:58:14.769060 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.769041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kp9x\" (UniqueName: \"kubernetes.io/projected/e67c6e70-a7b4-443e-b069-892373a430a1-kube-api-access-2kp9x\") pod \"node-resolver-s7489\" (UID: \"e67c6e70-a7b4-443e-b069-892373a430a1\") " pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.847649 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.847577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5"
Sep 15 20:58:14.859170 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.859147    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98b7q"
Sep 15 20:58:14.868352 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.868334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s7489"
Sep 15 20:58:14.876055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.876040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:14.883061 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.883038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:14.890360 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.890344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h57dk"
Sep 15 20:58:14.897878 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.897858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c684b"
Sep 15 20:58:14.904412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.904395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-977rj"
Sep 15 20:58:14.910392 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:14.910373    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tl79d"
Sep 15 20:58:15.164635 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.164541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:15.164635 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.164599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:15.164635 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.164619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164713    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164726    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164740    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164749    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164782    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:16.16476267 +0000 UTC m=+3.951130661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164800    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:16.164790538 +0000 UTC m=+3.951158526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164800    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:15.164883 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.164854    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:16.164836958 +0000 UTC m=+3.951204951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:15.494284 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.494262    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f95378e_7403_4add_a255_67713c570ade.slice/crio-c96b40b4bdce3695b91777fd05f374487374b536a0e9b9f86a891bff4a5c8b7b WatchSource:0}: Error finding container c96b40b4bdce3695b91777fd05f374487374b536a0e9b9f86a891bff4a5c8b7b: Status 404 returned error can't find the container with id c96b40b4bdce3695b91777fd05f374487374b536a0e9b9f86a891bff4a5c8b7b
Sep 15 20:58:15.495128 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.495107    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed722cd_2171_44aa_ac37_36725ac5c779.slice/crio-9e05d9038b93c4cb7668eec575e5156624aecac342bcd48feaa6f360ba47694e WatchSource:0}: Error finding container 9e05d9038b93c4cb7668eec575e5156624aecac342bcd48feaa6f360ba47694e: Status 404 returned error can't find the container with id 9e05d9038b93c4cb7668eec575e5156624aecac342bcd48feaa6f360ba47694e
Sep 15 20:58:15.497173 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.497128    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a19d1c_f032_4458_be72_a297de17625e.slice/crio-ff01de36d3a414441908cc2db1d630f3c5a5fcd328a5eb397a195e5e4d03cd74 WatchSource:0}: Error finding container ff01de36d3a414441908cc2db1d630f3c5a5fcd328a5eb397a195e5e4d03cd74: Status 404 returned error can't find the container with id ff01de36d3a414441908cc2db1d630f3c5a5fcd328a5eb397a195e5e4d03cd74
Sep 15 20:58:15.498032 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.497885    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f883507_b4c4_40b9_992e_c0bcb81cc20e.slice/crio-a4852d8318f5b83065e07ce31b048f7b5b4a9e402db6a4c647f795cde46da69e WatchSource:0}: Error finding container a4852d8318f5b83065e07ce31b048f7b5b4a9e402db6a4c647f795cde46da69e: Status 404 returned error can't find the container with id a4852d8318f5b83065e07ce31b048f7b5b4a9e402db6a4c647f795cde46da69e
Sep 15 20:58:15.501197 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.501174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67c6e70_a7b4_443e_b069_892373a430a1.slice/crio-f0123b71aa85c117f68051fb37a3888daab96d3e0350092bdf5eeac895b8f49b WatchSource:0}: Error finding container f0123b71aa85c117f68051fb37a3888daab96d3e0350092bdf5eeac895b8f49b: Status 404 returned error can't find the container with id f0123b71aa85c117f68051fb37a3888daab96d3e0350092bdf5eeac895b8f49b
Sep 15 20:58:15.501975 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.501950    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd315a5f_06a8_472b_8269_d87f8d05b72d.slice/crio-f531303327ed360d9521687a2e7eab67fc9fee29ae56596a3659f9b692308d45 WatchSource:0}: Error finding container f531303327ed360d9521687a2e7eab67fc9fee29ae56596a3659f9b692308d45: Status 404 returned error can't find the container with id f531303327ed360d9521687a2e7eab67fc9fee29ae56596a3659f9b692308d45
Sep 15 20:58:15.503556 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.503534    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2dd47b8_6d65_4982_9541_0e3293d59c85.slice/crio-e188758916ccf584d01dfaac34de3775119594866d16e6b8c43263506447d9bf WatchSource:0}: Error finding container e188758916ccf584d01dfaac34de3775119594866d16e6b8c43263506447d9bf: Status 404 returned error can't find the container with id e188758916ccf584d01dfaac34de3775119594866d16e6b8c43263506447d9bf
Sep 15 20:58:15.507996 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:58:15.507911    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e24f48_e8b0_4e51_8d64_2eb5bbf76bf7.slice/crio-f212816290fd7449868edf7f3333175dd3633e30a36c48e297f628308cdd932e WatchSource:0}: Error finding container f212816290fd7449868edf7f3333175dd3633e30a36c48e297f628308cdd932e: Status 404 returned error can't find the container with id f212816290fd7449868edf7f3333175dd3633e30a36c48e297f628308cdd932e
Sep 15 20:58:15.579503 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.579452    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 20:53:14 +0000 UTC" deadline="2028-02-13 14:48:49.182237583 +0000 UTC"
Sep 15 20:58:15.579503 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.579501    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12377h50m33.602739351s"
Sep 15 20:58:15.684442 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.684418    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:15.684598 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:15.684519    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:15.692821 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.692791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerStarted","Data":"9e05d9038b93c4cb7668eec575e5156624aecac342bcd48feaa6f360ba47694e"}
Sep 15 20:58:15.693967 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.693944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5" event={"ID":"0f95378e-7403-4add-a255-67713c570ade","Type":"ContainerStarted","Data":"c96b40b4bdce3695b91777fd05f374487374b536a0e9b9f86a891bff4a5c8b7b"}
Sep 15 20:58:15.695116 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.695097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"320572383ace28f69c9a2326b2f310d74bfcbbaaa8ba834fffeb1f4865edb587"}
Sep 15 20:58:15.696132 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.696109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c684b" event={"ID":"c2dd47b8-6d65-4982-9541-0e3293d59c85","Type":"ContainerStarted","Data":"e188758916ccf584d01dfaac34de3775119594866d16e6b8c43263506447d9bf"}
Sep 15 20:58:15.697109 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.697089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s7489" event={"ID":"e67c6e70-a7b4-443e-b069-892373a430a1","Type":"ContainerStarted","Data":"f0123b71aa85c117f68051fb37a3888daab96d3e0350092bdf5eeac895b8f49b"}
Sep 15 20:58:15.698921 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.698889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal" event={"ID":"4ed654b3d866e9d8cd5563c046da053f","Type":"ContainerStarted","Data":"7ba36ae367bcc39ce6db21077c8213a62300c7be3739164f953d8f60308fcc0f"}
Sep 15 20:58:15.700049 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.700031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-977rj" event={"ID":"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7","Type":"ContainerStarted","Data":"f212816290fd7449868edf7f3333175dd3633e30a36c48e297f628308cdd932e"}
Sep 15 20:58:15.701018 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.700999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zp94c" event={"ID":"bd315a5f-06a8-472b-8269-d87f8d05b72d","Type":"ContainerStarted","Data":"f531303327ed360d9521687a2e7eab67fc9fee29ae56596a3659f9b692308d45"}
Sep 15 20:58:15.701965 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.701937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98b7q" event={"ID":"6f883507-b4c4-40b9-992e-c0bcb81cc20e","Type":"ContainerStarted","Data":"a4852d8318f5b83065e07ce31b048f7b5b4a9e402db6a4c647f795cde46da69e"}
Sep 15 20:58:15.702954 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.702938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tl79d" event={"ID":"15a19d1c-f032-4458-be72-a297de17625e","Type":"ContainerStarted","Data":"ff01de36d3a414441908cc2db1d630f3c5a5fcd328a5eb397a195e5e4d03cd74"}
Sep 15 20:58:15.711951 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:15.711911    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-2.ec2.internal" podStartSLOduration=1.711900951 podStartE2EDuration="1.711900951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:15.71179937 +0000 UTC m=+3.498167378" watchObservedRunningTime="2026-09-15 20:58:15.711900951 +0000 UTC m=+3.498268984"
Sep 15 20:58:16.173802 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.173725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:16.173802 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.173769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:16.173802 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.173798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:16.174075 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.173920    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:16.174075 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.173976    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.173958045 +0000 UTC m=+5.960326041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:16.174371 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174348    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:16.174454 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174401    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.174386558 +0000 UTC m=+5.960754546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:16.174521 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174487    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:16.174521 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174502    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:16.174521 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174514    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:16.174695 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.174549    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.174536185 +0000 UTC m=+5.960904177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:16.580871 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.579671    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 20:53:14 +0000 UTC" deadline="2028-04-14 08:46:40.255594654 +0000 UTC"
Sep 15 20:58:16.580871 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.579711    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13835h48m23.675888259s"
Sep 15 20:58:16.684750 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.684721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:16.684901 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.684858    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:16.685281 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.685260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:16.685378 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:16.685354    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:16.719508 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.719474    2577 generic.go:356] "Generic (PLEG): container finished" podID="c25b0db8bec3d49e89def24b24b07505" containerID="ad3d95724beac1c3063d0b7d259aa8cda42c67ef988faa26b87da5e625742b53" exitCode=0
Sep 15 20:58:16.720324 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:16.720293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal" event={"ID":"c25b0db8bec3d49e89def24b24b07505","Type":"ContainerDied","Data":"ad3d95724beac1c3063d0b7d259aa8cda42c67ef988faa26b87da5e625742b53"}
Sep 15 20:58:17.684000 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:17.683931    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:17.684507 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:17.684118    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:17.750113 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:17.749398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal" event={"ID":"c25b0db8bec3d49e89def24b24b07505","Type":"ContainerStarted","Data":"7394ef364ba4afdf6174ff1e80d6382fa4ba5728153529ff88e2d6d33beae2f1"}
Sep 15 20:58:17.766085 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:17.766030    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-2.ec2.internal" podStartSLOduration=3.7660091060000003 podStartE2EDuration="3.766009106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:17.764994925 +0000 UTC m=+5.551362935" watchObservedRunningTime="2026-09-15 20:58:17.766009106 +0000 UTC m=+5.552377115"
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:18.189057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:18.189106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:18.189136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189252    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189312    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:22.18929441 +0000 UTC m=+9.975662413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189720    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189771    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:22.189754932 +0000 UTC m=+9.976122924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189849    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189865    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189877    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:18.189958 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.189921    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:22.189909643 +0000 UTC m=+9.976277630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:18.687148 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:18.687117    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:18.687611 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.687249    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:18.687684 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:18.687667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:18.687805 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:18.687781    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:19.684402 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:19.684361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:19.684718 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:19.684513    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:20.684609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:20.684394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:20.685415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:20.685184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:20.685415 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:20.685295    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:20.685415 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:20.684541    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:21.685068 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:21.684963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:21.685529 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:21.685092    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:22.224869 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:22.224831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:22.224869 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:22.224884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:22.225134 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:22.224915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:22.225134 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225069    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:22.225134 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225131    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:30.225110872 +0000 UTC m=+18.011478881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225437    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225446    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225466    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225480    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225518    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:30.225500213 +0000 UTC m=+18.011868204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:22.225539 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.225539    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:30.22552842 +0000 UTC m=+18.011896412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:22.686856 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:22.686829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:22.687263 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.686969    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:22.688284 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:22.687330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:22.688284 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:22.687419    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:23.684238 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:23.684202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:23.684395 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:23.684333    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:24.684610 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:24.684540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:24.685016 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:24.684710    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:24.685016 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:24.684801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:24.685016 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:24.684922    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:25.684675 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:25.684640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:25.685077 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:25.684749    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:26.684263 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:26.684237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:26.684443 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:26.684238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:26.684443 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:26.684349    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:26.684585 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:26.684448    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:27.683910 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:27.683877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:27.684355 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:27.683995    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:28.683954 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:28.683920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:28.684407 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:28.684057    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:28.684407 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:28.684143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:28.684407 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:28.684255    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:29.684358 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:29.684327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:29.684937 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:29.684432    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:30.284510 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:30.284463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:30.284510 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:30.284513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:30.284775 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:30.284576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:30.284775 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284652    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:30.284775 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284720    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:46.284703228 +0000 UTC m=+34.071071213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:30.284775 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284728    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:30.284959 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284789    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:46.284770688 +0000 UTC m=+34.071138692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:30.284959 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284797    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:30.284959 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284816    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:30.284959 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284828    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:30.284959 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.284874    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:58:46.284864117 +0000 UTC m=+34.071232103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:30.684173 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:30.684083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:30.684173 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:30.684126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:30.684377 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.684224    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:30.684377 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:30.684345    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:31.684294 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:31.684262    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:31.684461 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:31.684368    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:32.685149 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:32.685110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:32.685597 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:32.685212    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:32.685597 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:32.685274    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:32.685597 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:32.685284    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:33.684317 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:33.684293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:33.684416 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:33.684396    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:34.684249 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.684094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:34.684915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.684095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:34.684915 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:34.684328    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:34.684915 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:34.684417    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:34.786116 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.786076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-977rj" event={"ID":"26e24f48-e8b0-4e51-8d64-2eb5bbf76bf7","Type":"ContainerStarted","Data":"aa750e631e0e369ef51fb56617fb3e804f78b9b3aa23082284c33672546d8fe2"}
Sep 15 20:58:34.787823 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.787800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zp94c" event={"ID":"bd315a5f-06a8-472b-8269-d87f8d05b72d","Type":"ContainerStarted","Data":"9e6bf09946b09be1e589d5eafd36e825d1b89fbc2a2fd705caf114f3f457b20e"}
Sep 15 20:58:34.789591 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.789534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tl79d" event={"ID":"15a19d1c-f032-4458-be72-a297de17625e","Type":"ContainerStarted","Data":"b16a49ac8ae7ec389bc98478ad2175cf0037e73e9e7996916c15f2f83ab91022"}
Sep 15 20:58:34.791364 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.791341    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="f69246c5afdc241c40f34fb304d0219cc2119b6d70e165cb900968d9585aed13" exitCode=0
Sep 15 20:58:34.791476 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.791419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"f69246c5afdc241c40f34fb304d0219cc2119b6d70e165cb900968d9585aed13"}
Sep 15 20:58:34.793306 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.793285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5" event={"ID":"0f95378e-7403-4add-a255-67713c570ade","Type":"ContainerStarted","Data":"f7c8b481e00cd16f7129541bf57be6a5c6f06c9a233ce7b049045e52cbe17c1e"}
Sep 15 20:58:34.796515 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.796496    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 20:58:34.796965 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.796945    2577 generic.go:356] "Generic (PLEG): container finished" podID="4dc6e8d9-fc44-4d1a-b425-d24bcccb9277" containerID="73b6ff17cdfd179cf3dc3acc275b6defe3eb58dd4154b6d1f77b2ee388e93ce3" exitCode=1
Sep 15 20:58:34.797047 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797009    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"cc56a63ffe69f789847270d04a4556baf3a58438fe868e647ca8108ee14e6b3e"}
Sep 15 20:58:34.797103 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"66419a5c78897f734a6cb2a6511fca5798724cfef757b0a8d52b4a3ca392e986"}
Sep 15 20:58:34.797154 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"d4507e18ef7916eef1ea3fd2f46fddd9f57a234dcfbf06b39505116a5f06fa45"}
Sep 15 20:58:34.797154 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"add02ae8e0f3b32ada2b06a9a162d311df386a09836c6adb4a374b59a6848df8"}
Sep 15 20:58:34.797256 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerDied","Data":"73b6ff17cdfd179cf3dc3acc275b6defe3eb58dd4154b6d1f77b2ee388e93ce3"}
Sep 15 20:58:34.797298 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.797256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"2c3b321fd312f1a1a0544ad03e81b456e5990c72826c3cfb6f32eec6dbed0f5e"}
Sep 15 20:58:34.798851 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.798830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c684b" event={"ID":"c2dd47b8-6d65-4982-9541-0e3293d59c85","Type":"ContainerStarted","Data":"9c55bd92e54528959d6c6f73e1485415d1e3f0731a24b425363f9d272cb4941e"}
Sep 15 20:58:34.802619 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.802593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s7489" event={"ID":"e67c6e70-a7b4-443e-b069-892373a430a1","Type":"ContainerStarted","Data":"1fcbdf8e9a6915e6d9cfccc1f52a2d455e784fe6f51cb5654e8e6d205c9ae2af"}
Sep 15 20:58:34.804976 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.804927    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-977rj" podStartSLOduration=4.444856274 podStartE2EDuration="22.804909332s" totalImagesPullingTime="18.360053058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.803733435 +0000 UTC m=+22.590101444" watchObservedRunningTime="2026-09-15 20:58:34.804909332 +0000 UTC m=+22.591277342"
Sep 15 20:58:34.819517 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.819475    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c684b" podStartSLOduration=4.488073663 podStartE2EDuration="22.819463884s" totalImagesPullingTime="18.331390221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.819152824 +0000 UTC m=+22.605520834" watchObservedRunningTime="2026-09-15 20:58:34.819463884 +0000 UTC m=+22.605831888"
Sep 15 20:58:34.855335 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.855297    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tl79d" podStartSLOduration=4.1793299489999995 podStartE2EDuration="22.855283034s" totalImagesPullingTime="18.675953085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.855173282 +0000 UTC m=+22.641541313" watchObservedRunningTime="2026-09-15 20:58:34.855283034 +0000 UTC m=+22.641651039"
Sep 15 20:58:34.871670 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.871635    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s7489" podStartSLOduration=4.505737815 podStartE2EDuration="22.871623305s" totalImagesPullingTime="18.36588549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.871216224 +0000 UTC m=+22.657584232" watchObservedRunningTime="2026-09-15 20:58:34.871623305 +0000 UTC m=+22.657991313"
Sep 15 20:58:34.889309 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:34.889270    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zp94c" podStartSLOduration=12.187408681 podStartE2EDuration="22.889258712s" totalImagesPullingTime="10.701850031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.888848155 +0000 UTC m=+22.675216162" watchObservedRunningTime="2026-09-15 20:58:34.889258712 +0000 UTC m=+22.675626720"
Sep 15 20:58:35.249446 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.249260    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 20:58:35.614368 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.614270    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T20:58:35.249444141Z","UUID":"ce484f32-797c-4a2e-b4c3-4633d95fee73","Handler":null,"Name":"","Endpoint":""}
Sep 15 20:58:35.615972 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.615947    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 20:58:35.616116 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.615985    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 20:58:35.684934 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.684905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:35.685268 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:35.685012    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:35.806332 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.806291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98b7q" event={"ID":"6f883507-b4c4-40b9-992e-c0bcb81cc20e","Type":"ContainerStarted","Data":"01bc347c9b54552486738298daf3434a6f22f34a0a20907fedf20b6d945bbd95"}
Sep 15 20:58:35.809647 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:35.809609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5" event={"ID":"0f95378e-7403-4add-a255-67713c570ade","Type":"ContainerStarted","Data":"a27bb957e96d820d8ccd7590250dfa9fbfa688782c0e9242c863a13b127fd1ba"}
Sep 15 20:58:36.684608 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.684576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:36.684789 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:36.684696    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:36.684789 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.684582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:36.684905 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:36.684790    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:36.814460 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.814373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5" event={"ID":"0f95378e-7403-4add-a255-67713c570ade","Type":"ContainerStarted","Data":"36e88ee575d84f341ed09ee70ca2c7489c2df172ed9e652b896933d17626402d"}
Sep 15 20:58:36.818025 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.817997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 20:58:36.818458 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.818429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"a55296f6f1bed1d1815c1267e7d0091776864368e757fa08729587021a2c2e4a"}
Sep 15 20:58:36.830793 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:36.830755    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-98b7q" podStartSLOduration=6.493440653 podStartE2EDuration="24.83074077s" totalImagesPullingTime="18.337300117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:35.826804472 +0000 UTC m=+23.613172480" watchObservedRunningTime="2026-09-15 20:58:36.83074077 +0000 UTC m=+24.617108778"
Sep 15 20:58:37.684522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:37.684493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:37.684721 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:37.684631    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:38.687743 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:38.687715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:38.688127 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:38.687720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:38.688127 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:38.687838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:38.688127 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:38.687926    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:38.961064 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:38.960978    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:38.961703 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:38.961682    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:38.976330 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:38.976279    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hm6p5" podStartSLOduration=6.724826788 podStartE2EDuration="26.976265355s" totalImagesPullingTime="20.251438567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:36.845544668 +0000 UTC m=+24.631912675" watchObservedRunningTime="2026-09-15 20:58:38.976265355 +0000 UTC m=+26.762633379"
Sep 15 20:58:39.684496 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.684330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:39.684657 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:39.684620    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:39.827503 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.827464    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="a9f6a094fc73768c75cd6743060d140af10dda3fc3edeea35606ad9154a4dcc8" exitCode=0
Sep 15 20:58:39.828053 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.827545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"a9f6a094fc73768c75cd6743060d140af10dda3fc3edeea35606ad9154a4dcc8"}
Sep 15 20:58:39.831509 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.831489    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 20:58:39.831926 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.831904    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"8f12e0347a79c97a82861a901c9c77f34ff7cae221731496869ea98c99c250bc"}
Sep 15 20:58:39.832024 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.832003    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:39.832356 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.832336    2577 scope.go:117] "RemoveContainer" containerID="73b6ff17cdfd179cf3dc3acc275b6defe3eb58dd4154b6d1f77b2ee388e93ce3"
Sep 15 20:58:39.832544 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:39.832521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zp94c"
Sep 15 20:58:40.684118 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.684088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:40.684238 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:40.684192    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:40.684284 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.684260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:40.684412 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:40.684391    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:40.835681 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.835651    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="c0c5d713b02ffdc23cfff5b33b5d95098aab3f8524197e099c2f9d142f295f07" exitCode=0
Sep 15 20:58:40.836164 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.835734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"c0c5d713b02ffdc23cfff5b33b5d95098aab3f8524197e099c2f9d142f295f07"}
Sep 15 20:58:40.839640 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.839625    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 20:58:40.840034 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.840010    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" event={"ID":"4dc6e8d9-fc44-4d1a-b425-d24bcccb9277","Type":"ContainerStarted","Data":"293b8c1afb44e38275e3c6c6cd19b6208548ca227c20271e9c3d2d99cc295ca6"}
Sep 15 20:58:40.840176 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.840159    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:40.840250 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.840236    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:40.840403 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.840387    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:40.855493 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.855471    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:40.855629 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.855615    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:58:40.878930 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:40.878891    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-78t57" podStartSLOduration=10.480067634 podStartE2EDuration="28.878878453s" totalImagesPullingTime="18.398810819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:40.878877708 +0000 UTC m=+28.665245717" watchObservedRunningTime="2026-09-15 20:58:40.878878453 +0000 UTC m=+28.665246438"
Sep 15 20:58:41.684289 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:41.684214    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:41.684410 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:41.684312    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:42.685287 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:42.685252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:42.685845 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:42.685373    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:42.685845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:42.685415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:42.685845 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:42.685497    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:42.845512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:42.845484    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="c5c80398e57616fd92bfd0a506b1cb000b5bf987b50e720e78347799027bb309" exitCode=0
Sep 15 20:58:42.845701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:42.845606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"c5c80398e57616fd92bfd0a506b1cb000b5bf987b50e720e78347799027bb309"}
Sep 15 20:58:43.684609 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:43.684558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:43.684767 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:43.684707    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:44.684825 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:44.684787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:44.685227 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:44.684934    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:44.685355 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:44.685337    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:44.685446 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:44.685427    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:45.684734 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:45.684700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:45.684919 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:45.684827    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:46.310270 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:46.310240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:46.310525 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:46.310286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:46.310525 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310397    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:46.310525 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310457    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs podName:9a072b95-451a-4006-8b19-5ecf3ef181b0 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:18.310435466 +0000 UTC m=+66.096803453 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs") pod "network-metrics-daemon-s8465" (UID: "9a072b95-451a-4006-8b19-5ecf3ef181b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:46.310525 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310518    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310535    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310548    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2qs52 for pod openshift-network-diagnostics/network-check-target-9hj9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:46.310580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310610    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52 podName:eb4a6c98-49df-4014-a462-8eea0d53abff nodeName:}" failed. No retries permitted until 2026-09-15 20:59:18.310597251 +0000 UTC m=+66.096965260 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2qs52" (UniqueName: "kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52") pod "network-check-target-9hj9w" (UID: "eb4a6c98-49df-4014-a462-8eea0d53abff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310676    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:46.310777 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.310735    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret podName:723574f1-d85e-4b0e-9f37-9c0a96789c22 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:18.31071931 +0000 UTC m=+66.097087311 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret") pod "global-pull-secret-syncer-p4qc5" (UID: "723574f1-d85e-4b0e-9f37-9c0a96789c22") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:46.684968 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:46.684888    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:46.685381 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.685042    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:46.685381 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:46.685098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:46.685381 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:46.685192    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:47.684506 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:47.684440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:47.684666 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:47.684588    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:47.860674 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:47.860640    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="22856e907254b83f3b3df45addd2333ce828a3a5bea2e8251b670cb8ef414443" exitCode=0
Sep 15 20:58:47.861026 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:47.860702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"22856e907254b83f3b3df45addd2333ce828a3a5bea2e8251b670cb8ef414443"}
Sep 15 20:58:48.684463 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:48.684292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:48.684744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:48.684306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:48.684744 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:48.684539    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:48.684744 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:48.684648    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:48.865575 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:48.865528    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ed722cd-2171-44aa-ac37-36725ac5c779" containerID="139ef485cd442b5b8687b3c59da8f932323ffefd76cfe2ccbefa8ae8dca8d9bf" exitCode=0
Sep 15 20:58:48.865918 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:48.865599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerDied","Data":"139ef485cd442b5b8687b3c59da8f932323ffefd76cfe2ccbefa8ae8dca8d9bf"}
Sep 15 20:58:49.684132 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:49.684106    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:49.684303 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:49.684195    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:49.870834 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:49.870804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h57dk" event={"ID":"8ed722cd-2171-44aa-ac37-36725ac5c779","Type":"ContainerStarted","Data":"2ba287c762dd4994c76059d3559852d7f9f99cbda9b91c140a4f54ac7064c711"}
Sep 15 20:58:49.891821 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:49.891769    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h57dk" podStartSLOduration=8.586322663 podStartE2EDuration="37.891756003s" totalImagesPullingTime="29.30543334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:49.890145846 +0000 UTC m=+37.676513854" watchObservedRunningTime="2026-09-15 20:58:49.891756003 +0000 UTC m=+37.678124010"
Sep 15 20:58:50.684342 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:50.684315    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:50.684542 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:50.684411    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:50.684542 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:50.684503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:50.684686 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:50.684607    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:51.684057 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:51.684029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:51.684434 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:51.684122    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:52.684698 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:52.684670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:52.685131 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:52.684756    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:52.685131 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:52.684854    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:52.685131 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:52.685003    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:53.684154 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:53.684116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:53.684331 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:53.684222    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:54.684891 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:54.684853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:54.685259 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:54.684897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:54.685259 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:54.684974    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:54.685259 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:54.685086    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:55.684355 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:55.684322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:55.684587 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:55.684435    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:56.683931 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:56.683899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:56.683931 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:56.683912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:56.684299 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:56.684009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:56.684299 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:56.684144    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:57.684819 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:57.684787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:57.685228 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:57.684887    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:58:58.684148 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:58.684104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:58.684148 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:58.684134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:58.684379 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:58.684230    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:58.684379 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:58.684346    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:59.154359 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.154324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s8465"]
Sep 15 20:58:59.154831 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.154441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:58:59.154831 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:59.154584    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:58:59.160346 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.160307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hj9w"]
Sep 15 20:58:59.160673 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.160643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:58:59.160804 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:59.160762    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:58:59.161190 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.161164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4qc5"]
Sep 15 20:58:59.161300 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:58:59.161257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:58:59.161369 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:58:59.161347    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:59:00.684549 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:00.684509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:00.685049 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:00.684509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:00.685049 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:00.684640    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:59:00.685049 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:00.684509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:00.685049 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:00.684701    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:59:00.685049 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:00.684757    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:59:02.685639 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:02.685467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:02.686007 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:02.685537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:02.686007 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:02.685574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:02.686007 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:02.685695    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4qc5" podUID="723574f1-d85e-4b0e-9f37-9c0a96789c22"
Sep 15 20:59:02.686007 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:02.685829    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s8465" podUID="9a072b95-451a-4006-8b19-5ecf3ef181b0"
Sep 15 20:59:02.686007 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:02.685898    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hj9w" podUID="eb4a6c98-49df-4014-a462-8eea0d53abff"
Sep 15 20:59:03.035026 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.035002    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-2.ec2.internal" event="NodeReady"
Sep 15 20:59:03.035184 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.035098    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 20:59:03.076677 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.076652    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-86bbp"]
Sep 15 20:59:03.089082 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.089060    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tfjhm"]
Sep 15 20:59:03.089216 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.089196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.092285 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.092258    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p626r\""
Sep 15 20:59:03.092525 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.092496    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 20:59:03.092745 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.092725    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 20:59:03.102915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.102897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-86bbp"]
Sep 15 20:59:03.103007 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.102927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tfjhm"]
Sep 15 20:59:03.103007 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.102990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.106888 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.106861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 20:59:03.107139 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.106896    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wvdjj\""
Sep 15 20:59:03.107139 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.107107    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 20:59:03.107251 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.106877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 20:59:03.126046 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35b1de6c-9517-42bf-956d-32bca92ec8bd-cert\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.126046 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3dede8d-681d-4e4d-80c9-2539d00b566d-metrics-tls\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.126214 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48d6l\" (UniqueName: \"kubernetes.io/projected/35b1de6c-9517-42bf-956d-32bca92ec8bd-kube-api-access-48d6l\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.126214 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdgs\" (UniqueName: \"kubernetes.io/projected/b3dede8d-681d-4e4d-80c9-2539d00b566d-kube-api-access-vmdgs\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.126214 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3dede8d-681d-4e4d-80c9-2539d00b566d-tmp-dir\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.126332 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.126215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dede8d-681d-4e4d-80c9-2539d00b566d-config-volume\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.226623 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48d6l\" (UniqueName: \"kubernetes.io/projected/35b1de6c-9517-42bf-956d-32bca92ec8bd-kube-api-access-48d6l\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.226623 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdgs\" (UniqueName: \"kubernetes.io/projected/b3dede8d-681d-4e4d-80c9-2539d00b566d-kube-api-access-vmdgs\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.226808 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3dede8d-681d-4e4d-80c9-2539d00b566d-tmp-dir\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.226808 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dede8d-681d-4e4d-80c9-2539d00b566d-config-volume\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.226808 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35b1de6c-9517-42bf-956d-32bca92ec8bd-cert\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.226808 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3dede8d-681d-4e4d-80c9-2539d00b566d-metrics-tls\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.227030 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.226953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3dede8d-681d-4e4d-80c9-2539d00b566d-tmp-dir\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.227314 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.227292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dede8d-681d-4e4d-80c9-2539d00b566d-config-volume\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.230786 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.230757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3dede8d-681d-4e4d-80c9-2539d00b566d-metrics-tls\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.230876 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.230791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35b1de6c-9517-42bf-956d-32bca92ec8bd-cert\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.235351 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.235331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdgs\" (UniqueName: \"kubernetes.io/projected/b3dede8d-681d-4e4d-80c9-2539d00b566d-kube-api-access-vmdgs\") pod \"dns-default-86bbp\" (UID: \"b3dede8d-681d-4e4d-80c9-2539d00b566d\") " pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.235755 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.235734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48d6l\" (UniqueName: \"kubernetes.io/projected/35b1de6c-9517-42bf-956d-32bca92ec8bd-kube-api-access-48d6l\") pod \"ingress-canary-tfjhm\" (UID: \"35b1de6c-9517-42bf-956d-32bca92ec8bd\") " pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.299555 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.299501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:03.322139 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.322113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:03.322257 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.322200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.324888 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.324861    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 20:59:03.324888 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.324877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7k4mn\""
Sep 15 20:59:03.325049 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.324934    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 20:59:03.325219 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.325197    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 20:59:03.325321 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.325266    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 20:59:03.325321 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.325313    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 20:59:03.325526 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.325511    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 20:59:03.325822 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.325810    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 20:59:03.327175 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.327269 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.327314 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.327359 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb987\" (UniqueName: \"kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.327400 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.327439 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.327417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.402287 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.402259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:03.411389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.411360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tfjhm"
Sep 15 20:59:03.427799 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.427898 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.427898 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.427898 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.428024 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.428024 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.427938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb987\" (UniqueName: \"kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.428558 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.428493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.428687 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.428657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.428782 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.428763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.430367 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.430350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.430791 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.430771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.436219 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.436195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb987\" (UniqueName: \"kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987\") pod \"console-667bf55f74-b7hjz\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") " pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.631664 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.631587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:03.731964 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.731939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tfjhm"]
Sep 15 20:59:03.734656 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.734633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-86bbp"]
Sep 15 20:59:03.738309 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:03.738281    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b1de6c_9517_42bf_956d_32bca92ec8bd.slice/crio-876e5d0ce9b799253f20b5a5a0cd777ea4521aba197fdd0627689c3672ae9a7d WatchSource:0}: Error finding container 876e5d0ce9b799253f20b5a5a0cd777ea4521aba197fdd0627689c3672ae9a7d: Status 404 returned error can't find the container with id 876e5d0ce9b799253f20b5a5a0cd777ea4521aba197fdd0627689c3672ae9a7d
Sep 15 20:59:03.738702 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:03.738680    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3dede8d_681d_4e4d_80c9_2539d00b566d.slice/crio-3cfe74e62a9539a98e71e3d2bc7deee2f44a35f592b179d4ba3ebba8e7445b8f WatchSource:0}: Error finding container 3cfe74e62a9539a98e71e3d2bc7deee2f44a35f592b179d4ba3ebba8e7445b8f: Status 404 returned error can't find the container with id 3cfe74e62a9539a98e71e3d2bc7deee2f44a35f592b179d4ba3ebba8e7445b8f
Sep 15 20:59:03.901482 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.901401    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:03.903983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.903956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86bbp" event={"ID":"b3dede8d-681d-4e4d-80c9-2539d00b566d","Type":"ContainerStarted","Data":"3cfe74e62a9539a98e71e3d2bc7deee2f44a35f592b179d4ba3ebba8e7445b8f"}
Sep 15 20:59:03.905024 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:03.905004    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec05cfdb_648a_432e_b0d9_1c97cb50bc68.slice/crio-acaab16e6ce9c065fd11c051f0858356fc7cbfb644df4ee643891a4c848559d2 WatchSource:0}: Error finding container acaab16e6ce9c065fd11c051f0858356fc7cbfb644df4ee643891a4c848559d2: Status 404 returned error can't find the container with id acaab16e6ce9c065fd11c051f0858356fc7cbfb644df4ee643891a4c848559d2
Sep 15 20:59:03.905125 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:03.905077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tfjhm" event={"ID":"35b1de6c-9517-42bf-956d-32bca92ec8bd","Type":"ContainerStarted","Data":"876e5d0ce9b799253f20b5a5a0cd777ea4521aba197fdd0627689c3672ae9a7d"}
Sep 15 20:59:04.685323 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.685230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:04.685323 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.685284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:04.685858 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.685230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.688691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.688819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.689840    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.689864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.689884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-js7d5\""
Sep 15 20:59:04.692770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.690021    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r6tvw\""
Sep 15 20:59:04.910285 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:04.910240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-667bf55f74-b7hjz" event={"ID":"ec05cfdb-648a-432e-b0d9-1c97cb50bc68","Type":"ContainerStarted","Data":"acaab16e6ce9c065fd11c051f0858356fc7cbfb644df4ee643891a4c848559d2"}
Sep 15 20:59:08.661737 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.661707    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"]
Sep 15 20:59:08.663319 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.663305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:08.665751 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.665729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 15 20:59:08.665877 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.665759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rbgfs\""
Sep 15 20:59:08.671752 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.671732    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"]
Sep 15 20:59:08.765313 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.765276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qd4dh\" (UID: \"d088c154-2616-46d2-8f30-428d5f16c35f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:08.866639 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.866589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qd4dh\" (UID: \"d088c154-2616-46d2-8f30-428d5f16c35f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:08.866737 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:08.866713    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 15 20:59:08.866787 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:08.866777    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates podName:d088c154-2616-46d2-8f30-428d5f16c35f nodeName:}" failed. No retries permitted until 2026-09-15 20:59:09.366759859 +0000 UTC m=+57.153127844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-qd4dh" (UID: "d088c154-2616-46d2-8f30-428d5f16c35f") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 15 20:59:08.901853 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.901816    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zzwxd"]
Sep 15 20:59:08.903763 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.903742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.906626 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.906605    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 20:59:08.906854 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.906825    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 20:59:08.906854 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.906837    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 20:59:08.906975 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.906846    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-22kzx\""
Sep 15 20:59:08.906975 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.906927    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 20:59:08.916341 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.916290    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzwxd"]
Sep 15 20:59:08.925773 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.925750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86bbp" event={"ID":"b3dede8d-681d-4e4d-80c9-2539d00b566d","Type":"ContainerStarted","Data":"a6faef393c39efd9b3f52b5db6f09d04ca8623b9605ee868b879f396a8165f30"}
Sep 15 20:59:08.925915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.925799    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:08.925915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.925817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86bbp" event={"ID":"b3dede8d-681d-4e4d-80c9-2539d00b566d","Type":"ContainerStarted","Data":"65b421a7b4a57206fb4fa8642685d0c96061f65eff5a806b9d558c31945f5e0a"}
Sep 15 20:59:08.927234 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.927212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tfjhm" event={"ID":"35b1de6c-9517-42bf-956d-32bca92ec8bd","Type":"ContainerStarted","Data":"a70ef64797995185915483055af01cb01d4b36375f84bfe7e4050d5d7b91a260"}
Sep 15 20:59:08.928711 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.928689    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-667bf55f74-b7hjz" event={"ID":"ec05cfdb-648a-432e-b0d9-1c97cb50bc68","Type":"ContainerStarted","Data":"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"}
Sep 15 20:59:08.946553 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.946507    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-667bf55f74-b7hjz" podStartSLOduration=2.000722442 podStartE2EDuration="5.946491744s" totalImagesPullingTime="3.945769302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.946047846 +0000 UTC m=+56.732415866" watchObservedRunningTime="2026-09-15 20:59:08.946491744 +0000 UTC m=+56.732859746"
Sep 15 20:59:08.960684 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.960638    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tfjhm" podStartSLOduration=1.847662176 podStartE2EDuration="5.960623278s" totalImagesPullingTime="4.112961102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.959958325 +0000 UTC m=+56.746326333" watchObservedRunningTime="2026-09-15 20:59:08.960623278 +0000 UTC m=+56.746991285"
Sep 15 20:59:08.967299 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.967266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.967439 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.967419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc5c4d7-d584-4d43-aff2-1afe67162ca6-crio-socket\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.967535 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.967519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc5c4d7-d584-4d43-aff2-1afe67162ca6-data-volume\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.967611 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.967593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqx5\" (UniqueName: \"kubernetes.io/projected/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-api-access-4nqx5\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.967935 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.967902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:08.976957 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:08.976913    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-86bbp" podStartSLOduration=1.8745714260000002 podStartE2EDuration="5.976900467s" totalImagesPullingTime="4.102329041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.97617848 +0000 UTC m=+56.762546488" watchObservedRunningTime="2026-09-15 20:59:08.976900467 +0000 UTC m=+56.763268472"
Sep 15 20:59:09.068788 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc5c4d7-d584-4d43-aff2-1afe67162ca6-crio-socket\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.068982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc5c4d7-d584-4d43-aff2-1afe67162ca6-data-volume\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.068982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqx5\" (UniqueName: \"kubernetes.io/projected/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-api-access-4nqx5\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.068982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.068982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.068982 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.068938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edc5c4d7-d584-4d43-aff2-1afe67162ca6-crio-socket\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.069231 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.069026    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.069231 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.069025    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.069231 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.069095    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:09.569076292 +0000 UTC m=+57.355444280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.069231 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.069115    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:09.569105043 +0000 UTC m=+57.355473039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.069231 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.069177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edc5c4d7-d584-4d43-aff2-1afe67162ca6-data-volume\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.079880 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.079853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqx5\" (UniqueName: \"kubernetes.io/projected/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-api-access-4nqx5\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.370251 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.370216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qd4dh\" (UID: \"d088c154-2616-46d2-8f30-428d5f16c35f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:09.372631 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.372610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d088c154-2616-46d2-8f30-428d5f16c35f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-qd4dh\" (UID: \"d088c154-2616-46d2-8f30-428d5f16c35f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:09.571173 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.571131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:09.571357 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.571302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.571357 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.571341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:09.571466 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.571397    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.571466 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.571464    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:10.57144848 +0000 UTC m=+58.357816466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.571553 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.571499    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.571624 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:09.571559    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:10.571545828 +0000 UTC m=+58.357913814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.606848 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.606794    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86bbp_b3dede8d-681d-4e4d-80c9-2539d00b566d/dns/0.log"
Sep 15 20:59:09.779559 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.779526    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86bbp_b3dede8d-681d-4e4d-80c9-2539d00b566d/kube-rbac-proxy/0.log"
Sep 15 20:59:09.869662 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.869632    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"]
Sep 15 20:59:09.872823 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:09.872794    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd088c154_2616_46d2_8f30_428d5f16c35f.slice/crio-ee9601602d6933836a0444b704cffd68e0476f6c04e7f4175c3d1ad30ec709f3 WatchSource:0}: Error finding container ee9601602d6933836a0444b704cffd68e0476f6c04e7f4175c3d1ad30ec709f3: Status 404 returned error can't find the container with id ee9601602d6933836a0444b704cffd68e0476f6c04e7f4175c3d1ad30ec709f3
Sep 15 20:59:09.933088 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:09.932998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh" event={"ID":"d088c154-2616-46d2-8f30-428d5f16c35f","Type":"ContainerStarted","Data":"ee9601602d6933836a0444b704cffd68e0476f6c04e7f4175c3d1ad30ec709f3"}
Sep 15 20:59:10.579671 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:10.579633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:10.579671 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:10.579670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:10.580000 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:10.579761    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:10.580000 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:10.579779    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:10.580000 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:10.579829    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:12.579815286 +0000 UTC m=+60.366183272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:10.580000 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:10.579844    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:12.579837702 +0000 UTC m=+60.366205688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:11.179221 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.179184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s7489_e67c6e70-a7b4-443e-b069-892373a430a1/dns-node-resolver/0.log"
Sep 15 20:59:11.940018 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.939980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh" event={"ID":"d088c154-2616-46d2-8f30-428d5f16c35f","Type":"ContainerStarted","Data":"d09564056cd719a7611c81ea811fa686b6514bf3436bd4842d884d70be72c152"}
Sep 15 20:59:11.940217 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.940199    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:11.944832 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.944814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh"
Sep 15 20:59:11.954167 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.954123    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-qd4dh" podStartSLOduration=2.467742029 podStartE2EDuration="3.954111098s" totalImagesPullingTime="1.486369069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:11.953796818 +0000 UTC m=+59.740164828" watchObservedRunningTime="2026-09-15 20:59:11.954111098 +0000 UTC m=+59.740479106"
Sep 15 20:59:11.981404 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:11.981380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c684b_c2dd47b8-6d65-4982-9541-0e3293d59c85/node-ca/0.log"
Sep 15 20:59:12.595015 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:12.594976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:12.595015 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:12.595014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:12.595435 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:12.595101    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:12.595435 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:12.595111    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:12.595435 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:12.595167    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:16.595152666 +0000 UTC m=+64.381520651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:12.595435 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:12.595189    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:16.59518317 +0000 UTC m=+64.381551156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:12.857326 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:12.857256    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78t57"
Sep 15 20:59:13.179309 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.179226    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tfjhm_35b1de6c-9517-42bf-956d-32bca92ec8bd/serve-healthcheck-canary/0.log"
Sep 15 20:59:13.278046 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.278018    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 20:59:13.281925 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.281909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.288981 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.288954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 20:59:13.289779 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.289751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 20:59:13.300180 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300278 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300278 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300278 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdww\" (UniqueName: \"kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.300415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.300385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401017 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.400978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdww\" (UniqueName: \"kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.401225 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.401161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.402046 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.402022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.402158 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.402022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.402197 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.402173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.402245 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.402220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.403838 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.403815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.403838 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.403830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.409156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.409132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdww\" (UniqueName: \"kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww\") pod \"console-578f8f844-rzd2v\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") " pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.591002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.590969    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:13.632353 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.632310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:13.632796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.632399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:13.638462 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.638434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:13.859002 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.858926    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 20:59:13.862411 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:13.862387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1985a41_b4c3_4177_af1b_1f1e4208982c.slice/crio-e1f6b73aaff084a6e426af6ffb1a27dd37bc1160635e0451d5e1506774d3b068 WatchSource:0}: Error finding container e1f6b73aaff084a6e426af6ffb1a27dd37bc1160635e0451d5e1506774d3b068: Status 404 returned error can't find the container with id e1f6b73aaff084a6e426af6ffb1a27dd37bc1160635e0451d5e1506774d3b068
Sep 15 20:59:13.949085 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.949046    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f8f844-rzd2v" event={"ID":"e1985a41-b4c3-4177-af1b-1f1e4208982c","Type":"ContainerStarted","Data":"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"}
Sep 15 20:59:13.949283 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.949103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f8f844-rzd2v" event={"ID":"e1985a41-b4c3-4177-af1b-1f1e4208982c","Type":"ContainerStarted","Data":"e1f6b73aaff084a6e426af6ffb1a27dd37bc1160635e0451d5e1506774d3b068"}
Sep 15 20:59:13.953083 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.953054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:13.965485 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:13.965433    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-578f8f844-rzd2v" podStartSLOduration=0.965390041 podStartE2EDuration="965.390041ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:13.964451378 +0000 UTC m=+61.750819385" watchObservedRunningTime="2026-09-15 20:59:13.965390041 +0000 UTC m=+61.751758049"
Sep 15 20:59:16.628412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:16.628373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:16.628412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:16.628412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:16.628929 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:16.628498    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:16.628929 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:16.628513    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:16.628929 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:16.628603    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:24.62858331 +0000 UTC m=+72.414951314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:16.628929 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:16.628620    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls podName:edc5c4d7-d584-4d43-aff2-1afe67162ca6 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:24.628613818 +0000 UTC m=+72.414981804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzwxd" (UID: "edc5c4d7-d584-4d43-aff2-1afe67162ca6") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:17.086134 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.086100    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mzm68"]
Sep 15 20:59:17.110910 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.110875    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"]
Sep 15 20:59:17.111061 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.111001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.113434 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.113405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 20:59:17.113600 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.113486    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 20:59:17.113600 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.113520    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 20:59:17.114523 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.114502    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 20:59:17.114651 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.114547    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 20:59:17.114651 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.114627    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 20:59:17.114765 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.114653    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pcj6b\""
Sep 15 20:59:17.124720 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.124701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"]
Sep 15 20:59:17.124817 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.124789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.126983 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.126966    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 15 20:59:17.127086 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.126969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 15 20:59:17.127316 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.127299    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 15 20:59:17.127361 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.127315    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zbhn7\""
Sep 15 20:59:17.132138 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132243 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-sys\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132243 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132364 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-textfile\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132364 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-root\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132474 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132474 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pgtj\" (UniqueName: \"kubernetes.io/projected/f4251f66-c059-45fb-900a-4e918abfc965-kube-api-access-2pgtj\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132628 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-metrics-client-ca\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.132628 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.132540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-wtmp\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233244 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-wtmp\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-sys\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-sys\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8566m\" (UniqueName: \"kubernetes.io/projected/71185516-5e35-4581-8053-d7e65a26d39d-kube-api-access-8566m\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-wtmp\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233412 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-textfile\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.233481    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.233532    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls podName:f4251f66-c059-45fb-900a-4e918abfc965 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:17.733518863 +0000 UTC m=+65.519886849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls") pod "node-exporter-mzm68" (UID: "f4251f66-c059-45fb-900a-4e918abfc965") : secret "node-exporter-tls" not found
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-root\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f4251f66-c059-45fb-900a-4e918abfc965-root\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.233701 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/71185516-5e35-4581-8053-d7e65a26d39d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.234053 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.234053 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pgtj\" (UniqueName: \"kubernetes.io/projected/f4251f66-c059-45fb-900a-4e918abfc965-kube-api-access-2pgtj\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.234053 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-textfile\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.234053 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.233825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-metrics-client-ca\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.236145 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.236125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.243080 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.243052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.243213 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.243107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4251f66-c059-45fb-900a-4e918abfc965-metrics-client-ca\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.245339 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.245316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pgtj\" (UniqueName: \"kubernetes.io/projected/f4251f66-c059-45fb-900a-4e918abfc965-kube-api-access-2pgtj\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.334558 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.334558 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8566m\" (UniqueName: \"kubernetes.io/projected/71185516-5e35-4581-8053-d7e65a26d39d-kube-api-access-8566m\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.334812 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.334812 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.334725    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 15 20:59:17.334812 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.334781    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls podName:71185516-5e35-4581-8053-d7e65a26d39d nodeName:}" failed. No retries permitted until 2026-09-15 20:59:17.834765504 +0000 UTC m=+65.621133512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-d82l7" (UID: "71185516-5e35-4581-8053-d7e65a26d39d") : secret "kube-state-metrics-tls" not found
Sep 15 20:59:17.334812 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.334812 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.335033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.334837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/71185516-5e35-4581-8053-d7e65a26d39d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.335193 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.335172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/71185516-5e35-4581-8053-d7e65a26d39d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.335363 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.335346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.335435 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.335417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.342964 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.342907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.343287 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.343266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8566m\" (UniqueName: \"kubernetes.io/projected/71185516-5e35-4581-8053-d7e65a26d39d-kube-api-access-8566m\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.737751 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.737716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.740155 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.740119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f4251f66-c059-45fb-900a-4e918abfc965-node-exporter-tls\") pod \"node-exporter-mzm68\" (UID: \"f4251f66-c059-45fb-900a-4e918abfc965\") " pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:17.838478 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:17.838439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:17.838654 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.838558    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 15 20:59:17.838654 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:17.838649    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls podName:71185516-5e35-4581-8053-d7e65a26d39d nodeName:}" failed. No retries permitted until 2026-09-15 20:59:18.838632252 +0000 UTC m=+66.625000240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-d82l7" (UID: "71185516-5e35-4581-8053-d7e65a26d39d") : secret "kube-state-metrics-tls" not found
Sep 15 20:59:18.021303 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.021213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzm68"
Sep 15 20:59:18.060163 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:18.060136    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4251f66_c059_45fb_900a_4e918abfc965.slice/crio-d2e38f42277b7e1ea59ff3422138a74e022bb8a09b3d4a46a0e878443346350d WatchSource:0}: Error finding container d2e38f42277b7e1ea59ff3422138a74e022bb8a09b3d4a46a0e878443346350d: Status 404 returned error can't find the container with id d2e38f42277b7e1ea59ff3422138a74e022bb8a09b3d4a46a0e878443346350d
Sep 15 20:59:18.342176 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.342091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:18.342176 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.342168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:18.342373 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.342195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:18.344779 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.344751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:59:18.344907 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.344782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:59:18.345921 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.345905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:59:18.354970 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.354944    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:59:18.355223 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.355202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/723574f1-d85e-4b0e-9f37-9c0a96789c22-original-pull-secret\") pod \"global-pull-secret-syncer-p4qc5\" (UID: \"723574f1-d85e-4b0e-9f37-9c0a96789c22\") " pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:18.355296 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.355238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a072b95-451a-4006-8b19-5ecf3ef181b0-metrics-certs\") pod \"network-metrics-daemon-s8465\" (UID: \"9a072b95-451a-4006-8b19-5ecf3ef181b0\") " pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:18.365227 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.365197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qs52\" (UniqueName: \"kubernetes.io/projected/eb4a6c98-49df-4014-a462-8eea0d53abff-kube-api-access-2qs52\") pod \"network-check-target-9hj9w\" (UID: \"eb4a6c98-49df-4014-a462-8eea0d53abff\") " pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:18.503702 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.503673    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r6tvw\""
Sep 15 20:59:18.511770 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.511745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s8465"
Sep 15 20:59:18.512786 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.512763    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-js7d5\""
Sep 15 20:59:18.520773 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.520713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4qc5"
Sep 15 20:59:18.521830 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.521808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:18.787806 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.787772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s8465"]
Sep 15 20:59:18.791242 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:18.791208    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a072b95_451a_4006_8b19_5ecf3ef181b0.slice/crio-f812a2f03479312b769014efca47d0f206c99d720b11170af42fd71b796499a7 WatchSource:0}: Error finding container f812a2f03479312b769014efca47d0f206c99d720b11170af42fd71b796499a7: Status 404 returned error can't find the container with id f812a2f03479312b769014efca47d0f206c99d720b11170af42fd71b796499a7
Sep 15 20:59:18.811453 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.811422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hj9w"]
Sep 15 20:59:18.814152 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.814127    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4qc5"]
Sep 15 20:59:18.814584 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:18.814543    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4a6c98_49df_4014_a462_8eea0d53abff.slice/crio-aa09a9e0e7024d56fe8ada8ed93bf69c00ad4ae7fe7348cb8f8db615ede3a333 WatchSource:0}: Error finding container aa09a9e0e7024d56fe8ada8ed93bf69c00ad4ae7fe7348cb8f8db615ede3a333: Status 404 returned error can't find the container with id aa09a9e0e7024d56fe8ada8ed93bf69c00ad4ae7fe7348cb8f8db615ede3a333
Sep 15 20:59:18.817482 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:18.817460    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723574f1_d85e_4b0e_9f37_9c0a96789c22.slice/crio-f549a931df4e5835d09a0080c510573ebd1d92b63b7c015f81a8caf8c08ac3a7 WatchSource:0}: Error finding container f549a931df4e5835d09a0080c510573ebd1d92b63b7c015f81a8caf8c08ac3a7: Status 404 returned error can't find the container with id f549a931df4e5835d09a0080c510573ebd1d92b63b7c015f81a8caf8c08ac3a7
Sep 15 20:59:18.848789 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.848707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:18.851539 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.851518    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/71185516-5e35-4581-8053-d7e65a26d39d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-d82l7\" (UID: \"71185516-5e35-4581-8053-d7e65a26d39d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:18.932464 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.932426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"
Sep 15 20:59:18.935670 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.935651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-86bbp"
Sep 15 20:59:18.967845 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.967811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hj9w" event={"ID":"eb4a6c98-49df-4014-a462-8eea0d53abff","Type":"ContainerStarted","Data":"aa09a9e0e7024d56fe8ada8ed93bf69c00ad4ae7fe7348cb8f8db615ede3a333"}
Sep 15 20:59:18.969204 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.969179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzm68" event={"ID":"f4251f66-c059-45fb-900a-4e918abfc965","Type":"ContainerStarted","Data":"d2e38f42277b7e1ea59ff3422138a74e022bb8a09b3d4a46a0e878443346350d"}
Sep 15 20:59:18.970464 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.970436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4qc5" event={"ID":"723574f1-d85e-4b0e-9f37-9c0a96789c22","Type":"ContainerStarted","Data":"f549a931df4e5835d09a0080c510573ebd1d92b63b7c015f81a8caf8c08ac3a7"}
Sep 15 20:59:18.971698 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:18.971676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8465" event={"ID":"9a072b95-451a-4006-8b19-5ecf3ef181b0","Type":"ContainerStarted","Data":"f812a2f03479312b769014efca47d0f206c99d720b11170af42fd71b796499a7"}
Sep 15 20:59:19.344308 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:19.344275    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7"]
Sep 15 20:59:19.349066 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:19.348997    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71185516_5e35_4581_8053_d7e65a26d39d.slice/crio-f4aa847a4467a972141c066ba762b58835219ab78a0282e836e5d67011e75fde WatchSource:0}: Error finding container f4aa847a4467a972141c066ba762b58835219ab78a0282e836e5d67011e75fde: Status 404 returned error can't find the container with id f4aa847a4467a972141c066ba762b58835219ab78a0282e836e5d67011e75fde
Sep 15 20:59:19.978389 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:19.978348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7" event={"ID":"71185516-5e35-4581-8053-d7e65a26d39d","Type":"ContainerStarted","Data":"f4aa847a4467a972141c066ba762b58835219ab78a0282e836e5d67011e75fde"}
Sep 15 20:59:19.982410 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:19.981738    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4251f66-c059-45fb-900a-4e918abfc965" containerID="439a9bf013ed4a55c2328d6feae46584a4c4d4487a4d71a721dab11f690f816d" exitCode=0
Sep 15 20:59:19.982410 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:19.981795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzm68" event={"ID":"f4251f66-c059-45fb-900a-4e918abfc965","Type":"ContainerDied","Data":"439a9bf013ed4a55c2328d6feae46584a4c4d4487a4d71a721dab11f690f816d"}
Sep 15 20:59:20.989431 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:20.989327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8465" event={"ID":"9a072b95-451a-4006-8b19-5ecf3ef181b0","Type":"ContainerStarted","Data":"5cf1b19db4ab678574e493a8121782f9878afcb846dbddab13fa26a76297aeda"}
Sep 15 20:59:20.989431 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:20.989391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s8465" event={"ID":"9a072b95-451a-4006-8b19-5ecf3ef181b0","Type":"ContainerStarted","Data":"63c190f25d6e86ab448f41a4e0876a3a6ea0623fdcb7d80ab178fc37046bd744"}
Sep 15 20:59:20.993178 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:20.992559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzm68" event={"ID":"f4251f66-c059-45fb-900a-4e918abfc965","Type":"ContainerStarted","Data":"cb3ca2718fc2465d84a20165e4c302b85c81c36a658c36ff19e11556d66f9f4d"}
Sep 15 20:59:20.993178 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:20.992637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzm68" event={"ID":"f4251f66-c059-45fb-900a-4e918abfc965","Type":"ContainerStarted","Data":"85ccc9a2fbf7cc7f4e881c6aa5bcd536c0ecbd1d9bf64984f93ad7fc905d164c"}
Sep 15 20:59:21.006045 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.005991    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s8465" podStartSLOduration=67.437230495 podStartE2EDuration="1m9.005970696s" totalImagesPullingTime="1.568740201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:21.004246437 +0000 UTC m=+68.790614446" watchObservedRunningTime="2026-09-15 20:59:21.005970696 +0000 UTC m=+68.792338703"
Sep 15 20:59:21.026119 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.026063    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mzm68" podStartSLOduration=2.917983368 podStartE2EDuration="4.026043983s" totalImagesPullingTime="1.036397035s" totalInitContainerRuntime="71.66358ms" isStatefulPod=true podCreationTimestamp="2026-09-15 20:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:21.025000644 +0000 UTC m=+68.811368652" watchObservedRunningTime="2026-09-15 20:59:21.026043983 +0000 UTC m=+68.812411991"
Sep 15 20:59:21.073703 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.073680    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d59576f75-txgmp"]
Sep 15 20:59:21.077533 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.077508    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.080720 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.080241    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-82ifmior9gqbc\""
Sep 15 20:59:21.080720 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.080522    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gw6fk\""
Sep 15 20:59:21.081239 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.081219    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 20:59:21.081512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.081492    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 20:59:21.082305 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.081799    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 20:59:21.082756 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.082733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 20:59:21.082853 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.082782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 20:59:21.099006 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.098889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d59576f75-txgmp"]
Sep 15 20:59:21.170656 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-grpc-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.170764 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.170764 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e93dcb8-84fa-4add-915c-37b307b0c7a5-metrics-client-ca\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.170878 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.170878 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.170878 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt49m\" (UniqueName: \"kubernetes.io/projected/9e93dcb8-84fa-4add-915c-37b307b0c7a5-kube-api-access-dt49m\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.171014 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.171014 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.170939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271453 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271627 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e93dcb8-84fa-4add-915c-37b307b0c7a5-metrics-client-ca\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271627 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271627 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt49m\" (UniqueName: \"kubernetes.io/projected/9e93dcb8-84fa-4add-915c-37b307b0c7a5-kube-api-access-dt49m\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.271940 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.271817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-grpc-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.275481 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.274586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e93dcb8-84fa-4add-915c-37b307b0c7a5-metrics-client-ca\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.275481 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.275051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-grpc-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.283657 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.280290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.283657 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.281429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-tls\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.283657 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.282220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.288968 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.288205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt49m\" (UniqueName: \"kubernetes.io/projected/9e93dcb8-84fa-4add-915c-37b307b0c7a5-kube-api-access-dt49m\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.289097 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.289008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.289646 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.289620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9e93dcb8-84fa-4add-915c-37b307b0c7a5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d59576f75-txgmp\" (UID: \"9e93dcb8-84fa-4add-915c-37b307b0c7a5\") " pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.415033 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.414756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:21.710431 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:21.710363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d59576f75-txgmp"]
Sep 15 20:59:22.000943 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.000312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7" event={"ID":"71185516-5e35-4581-8053-d7e65a26d39d","Type":"ContainerStarted","Data":"8bba6591944ac092e3619673e6ec8194dc7ea0b02326e5e66e13e06d1df33f55"}
Sep 15 20:59:22.000943 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.000385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7" event={"ID":"71185516-5e35-4581-8053-d7e65a26d39d","Type":"ContainerStarted","Data":"fb907d7968e2a2eff278a031ee8695cd86261da84c8878e2af61cd56054f59f9"}
Sep 15 20:59:22.000943 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.000406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7" event={"ID":"71185516-5e35-4581-8053-d7e65a26d39d","Type":"ContainerStarted","Data":"b7bd2d032682c0c7b8b106e177c6c6b6330a42a8faf5e09b4d8a2521c0271abf"}
Sep 15 20:59:22.021723 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.021665    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-d82l7" podStartSLOduration=3.307378291 podStartE2EDuration="5.021641914s" totalImagesPullingTime="1.714263623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:22.020287908 +0000 UTC m=+69.806655917" watchObservedRunningTime="2026-09-15 20:59:22.021641914 +0000 UTC m=+69.808009924"
Sep 15 20:59:22.293627 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.293524    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"]
Sep 15 20:59:22.298603 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.298576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.301391 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 15 20:59:22.301534 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-nj6xw\""
Sep 15 20:59:22.301534 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301491    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 15 20:59:22.301668 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301428    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 15 20:59:22.301876 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 15 20:59:22.301876 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.301785    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 15 20:59:22.306925 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.306899    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 15 20:59:22.310764 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.310648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"]
Sep 15 20:59:22.382682 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-federate-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-serving-certs-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-metrics-client-ca\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gkrj\" (UniqueName: \"kubernetes.io/projected/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-kube-api-access-7gkrj\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.382875 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.383110 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.382958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484230 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484423 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-federate-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484423 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-serving-certs-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484544 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-metrics-client-ca\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484544 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gkrj\" (UniqueName: \"kubernetes.io/projected/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-kube-api-access-7gkrj\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484544 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484713 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.484713 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.484648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.485181 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.485151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-serving-certs-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.485322 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.485296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-metrics-client-ca\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.485802 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.485758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.487589 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.487538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-telemeter-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.487792 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.487746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.487792 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.487764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.487943 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.487855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-federate-client-tls\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.492442 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.492420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gkrj\" (UniqueName: \"kubernetes.io/projected/8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d-kube-api-access-7gkrj\") pod \"telemeter-client-5967c7fc7-9zkp9\" (UID: \"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d\") " pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.612787 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:22.612709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"
Sep 15 20:59:22.757242 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:22.757206    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e93dcb8_84fa_4add_915c_37b307b0c7a5.slice/crio-4f64965a862e9d2b84f17c51c26671e740db6148fad05109a26d7db4124fd363 WatchSource:0}: Error finding container 4f64965a862e9d2b84f17c51c26671e740db6148fad05109a26d7db4124fd363: Status 404 returned error can't find the container with id 4f64965a862e9d2b84f17c51c26671e740db6148fad05109a26d7db4124fd363
Sep 15 20:59:23.006579 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.006534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"4f64965a862e9d2b84f17c51c26671e740db6148fad05109a26d7db4124fd363"}
Sep 15 20:59:23.327843 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.327756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:59:23.331608 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.331587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.335262 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.335235    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 20:59:23.335395 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.335278    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336200    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b6q4l59fg8edm\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336233    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-667nd\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336472    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336506    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336635    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336679    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 20:59:23.337156 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.336947    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 20:59:23.337732 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.337192    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 20:59:23.337732 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.337596    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 20:59:23.341896 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.340379    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 20:59:23.341896 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.341657    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 20:59:23.345734 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.345709    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:59:23.392457 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392457 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.392744 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config-out\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgxtm\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-kube-api-access-vgxtm\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.392957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393115 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-web-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.393512 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.393193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494092 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-web-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494277 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config-out\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.494522 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgxtm\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-kube-api-access-vgxtm\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.496248 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.494548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.496248 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.495380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.496248 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.495416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.496248 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.495913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.496248 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.496131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.498501 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.496777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.498501 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.497686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.498501 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.498167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.498501 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.498183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.498501 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.498449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.499041 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.498820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.500288 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.500259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.500400 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.500343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.500718 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.500697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.500996 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.500974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.501180 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.501156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.501298 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.501273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.501384 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.501333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.502554 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.502533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-config-out\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.502666 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.502646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.503270 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.503247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.505973 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.505955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-web-config\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.508172 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.508149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgxtm\" (UniqueName: \"kubernetes.io/projected/19f9e565-a82b-4e3c-b9a8-57acb8a9757f-kube-api-access-vgxtm\") pod \"prometheus-k8s-0\" (UID: \"19f9e565-a82b-4e3c-b9a8-57acb8a9757f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:23.591340 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.591261    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:23.591340 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.591335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:23.597341 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.597315    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:23.644198 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:23.644164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:24.023184 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.023002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 20:59:24.071070 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.071034    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:24.596441 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.596411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5967c7fc7-9zkp9"]
Sep 15 20:59:24.603128 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.603015    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:59:24.614152 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:24.614123    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f9e565_a82b_4e3c_b9a8_57acb8a9757f.slice/crio-94fb2216a7e85e5c9ded24f50365d5d506b16acc6ad7102a005231195b7756d1 WatchSource:0}: Error finding container 94fb2216a7e85e5c9ded24f50365d5d506b16acc6ad7102a005231195b7756d1: Status 404 returned error can't find the container with id 94fb2216a7e85e5c9ded24f50365d5d506b16acc6ad7102a005231195b7756d1
Sep 15 20:59:24.708523 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.708490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:24.708738 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.708612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:24.709195 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.709174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edc5c4d7-d584-4d43-aff2-1afe67162ca6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:24.711514 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.711489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edc5c4d7-d584-4d43-aff2-1afe67162ca6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzwxd\" (UID: \"edc5c4d7-d584-4d43-aff2-1afe67162ca6\") " pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:24.815509 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.815449    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-22kzx\""
Sep 15 20:59:24.822975 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:24.822945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzwxd"
Sep 15 20:59:25.016915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.016871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4qc5" event={"ID":"723574f1-d85e-4b0e-9f37-9c0a96789c22","Type":"ContainerStarted","Data":"21212729190f127996092e60db8896b29c8583f728ed94402393eec296256874"}
Sep 15 20:59:25.018150 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.018119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9" event={"ID":"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d","Type":"ContainerStarted","Data":"fbb7d63b7bb292e51a0023ea216c8d1da15dd792ed47fa2c5537016e8dc80e8a"}
Sep 15 20:59:25.019443 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.019418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"94fb2216a7e85e5c9ded24f50365d5d506b16acc6ad7102a005231195b7756d1"}
Sep 15 20:59:25.021216 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.021196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hj9w" event={"ID":"eb4a6c98-49df-4014-a462-8eea0d53abff","Type":"ContainerStarted","Data":"0d76ae9a9e1360945b55bca2adf2b6f7fa16f1aef641d445e77d379f5cfec49e"}
Sep 15 20:59:25.048876 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.048798    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9hj9w" podStartSLOduration=67.547807721 podStartE2EDuration="1m13.048779484s" totalImagesPullingTime="5.500971763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:25.047876507 +0000 UTC m=+72.834244516" watchObservedRunningTime="2026-09-15 20:59:25.048779484 +0000 UTC m=+72.835147486"
Sep 15 20:59:25.049272 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.049241    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p4qc5" podStartSLOduration=67.540136775 podStartE2EDuration="1m13.049232743s" totalImagesPullingTime="5.509095968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:25.033641304 +0000 UTC m=+72.820009385" watchObservedRunningTime="2026-09-15 20:59:25.049232743 +0000 UTC m=+72.835600748"
Sep 15 20:59:25.104316 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:25.104287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzwxd"]
Sep 15 20:59:25.107332 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:25.107306    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc5c4d7_d584_4d43_aff2_1afe67162ca6.slice/crio-207c9341dd5e07a4a2ee68d8ddc03456c7bef4cadf94ee8f6b62c05d134babce WatchSource:0}: Error finding container 207c9341dd5e07a4a2ee68d8ddc03456c7bef4cadf94ee8f6b62c05d134babce: Status 404 returned error can't find the container with id 207c9341dd5e07a4a2ee68d8ddc03456c7bef4cadf94ee8f6b62c05d134babce
Sep 15 20:59:26.030516 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.030477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"3c411ff6dbbbc4f3edbf3b94f1501f7b904b671dfd23a25feb4ab467d8ee8571"}
Sep 15 20:59:26.032910 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.032749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzwxd" event={"ID":"edc5c4d7-d584-4d43-aff2-1afe67162ca6","Type":"ContainerStarted","Data":"819a80d04a9ec07b91a4d6a27ebfc749f311b5d1b0ad0ca1c9c512a18ab8a0ea"}
Sep 15 20:59:26.032910 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.032821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzwxd" event={"ID":"edc5c4d7-d584-4d43-aff2-1afe67162ca6","Type":"ContainerStarted","Data":"207c9341dd5e07a4a2ee68d8ddc03456c7bef4cadf94ee8f6b62c05d134babce"}
Sep 15 20:59:26.033234 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.033214    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 20:59:26.887288 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.887119    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:26.891981 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.891962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.903607 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.901746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:26.932676 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.932796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.932796 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.932903 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2mc\" (UniqueName: \"kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.932952 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.933001 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:26.933001 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:26.932983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2mc\" (UniqueName: \"kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.033807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.034867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.034917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.035345 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.035196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.036490 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.036470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.036750 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.036727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.037312 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.037292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.039705 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.039681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"455d9939f4e05ff8089e1445feb838b7baf03443069ceecfb0b5b6bb2ff45a7e"}
Sep 15 20:59:27.041803 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.041782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzwxd" event={"ID":"edc5c4d7-d584-4d43-aff2-1afe67162ca6","Type":"ContainerStarted","Data":"a62f55f5af0c949b776c9f68804ed4fa84174dfd7538eeb95f78e5b0b740febe"}
Sep 15 20:59:27.041956 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.041940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2mc\" (UniqueName: \"kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc\") pod \"console-6b5976bc54-zb897\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") " pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.043384 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.043359    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9" event={"ID":"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d","Type":"ContainerStarted","Data":"9df814bb987c563e6159bc0bf9acf18da4b3faafe5d5703edd04ceb66dbd84a8"}
Sep 15 20:59:27.044930 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.044908    2577 generic.go:356] "Generic (PLEG): container finished" podID="19f9e565-a82b-4e3c-b9a8-57acb8a9757f" containerID="a2eb327b420040878680a2bbde07620cab0a1d13feb9a987802b075872811c69" exitCode=0
Sep 15 20:59:27.045013 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.044993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerDied","Data":"a2eb327b420040878680a2bbde07620cab0a1d13feb9a987802b075872811c69"}
Sep 15 20:59:27.205330 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.205299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:27.475986 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:27.475900    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:27.478985 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:27.478960    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80701e5a_aff3_4336_aecb_4d35dbcf6b31.slice/crio-8bb66b64e3f25fa49ee48b4ca7e2957e54c359c7657dedba0fbdc709455d080f WatchSource:0}: Error finding container 8bb66b64e3f25fa49ee48b4ca7e2957e54c359c7657dedba0fbdc709455d080f: Status 404 returned error can't find the container with id 8bb66b64e3f25fa49ee48b4ca7e2957e54c359c7657dedba0fbdc709455d080f
Sep 15 20:59:28.052367 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.052340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5976bc54-zb897" event={"ID":"80701e5a-aff3-4336-aecb-4d35dbcf6b31","Type":"ContainerStarted","Data":"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"}
Sep 15 20:59:28.052781 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.052386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5976bc54-zb897" event={"ID":"80701e5a-aff3-4336-aecb-4d35dbcf6b31","Type":"ContainerStarted","Data":"8bb66b64e3f25fa49ee48b4ca7e2957e54c359c7657dedba0fbdc709455d080f"}
Sep 15 20:59:28.056169 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.056144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"3225af94050dbdb27c3f134f15284b3b3fc222c633d1bb27852aa9c9912001da"}
Sep 15 20:59:28.056292 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.056220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"b2d09556b687cb0d0ed54e67d5cd7f16bb004e952cea1d1645899e7113237b45"}
Sep 15 20:59:28.056292 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.056261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"8634f8cf7ba56aee6c26a200c8d8b409f39070adeb9ff0e439564fe66e31012e"}
Sep 15 20:59:28.059662 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.059639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9" event={"ID":"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d","Type":"ContainerStarted","Data":"d999a3fcc64c78d31d02e329f2b79be49db85c42a2d581dd0682926646004214"}
Sep 15 20:59:28.059769 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.059695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9" event={"ID":"8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d","Type":"ContainerStarted","Data":"25a49ac6dbbc33d2c252cf2762d99278a0c3bc5909eeabbe0116084a8da3f77c"}
Sep 15 20:59:28.080852 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.080727    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b5976bc54-zb897" podStartSLOduration=2.080701255 podStartE2EDuration="2.080701255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:28.073990934 +0000 UTC m=+75.860358942" watchObservedRunningTime="2026-09-15 20:59:28.080701255 +0000 UTC m=+75.867069351"
Sep 15 20:59:28.101065 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.099817    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5967c7fc7-9zkp9" podStartSLOduration=3.8929267149999998 podStartE2EDuration="6.099793877s" totalImagesPullingTime="2.206867162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:28.098807261 +0000 UTC m=+75.885175270" watchObservedRunningTime="2026-09-15 20:59:28.099793877 +0000 UTC m=+75.886161886"
Sep 15 20:59:28.533056 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.533016    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:28.567624 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.567586    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 20:59:28.572238 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.572207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.584198 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.584167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 20:59:28.647915 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.647873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648096 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.647929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648096 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.647999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648096 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.648049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648096 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.648077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648269 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.648214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.648269 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.648263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bdhp\" (UniqueName: \"kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749245 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749415 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749551 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bdhp\" (UniqueName: \"kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.749551 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.749489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.750143 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.750110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.750305 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.750278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.750426 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.750407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.750859 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.750816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.752518 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.752492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.753264 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.753229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.761401 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.761373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bdhp\" (UniqueName: \"kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp\") pod \"console-5479955bc7-87qv2\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") " pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:28.887180 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:28.886975    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:29.069181 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.069095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" event={"ID":"9e93dcb8-84fa-4add-915c-37b307b0c7a5","Type":"ContainerStarted","Data":"741ccea51b720cc275f0603114abf6cd209b1f7fb4fad35e2d5d89754bd4f245"}
Sep 15 20:59:29.069672 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.069248    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:29.071777 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.071749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzwxd" event={"ID":"edc5c4d7-d584-4d43-aff2-1afe67162ca6","Type":"ContainerStarted","Data":"e77b70637e38741f86057b2b6282794e7a19819af7c44169922dcc9949ab3dc9"}
Sep 15 20:59:29.091747 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.091691    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp" podStartSLOduration=4.234139436 podStartE2EDuration="8.091669249s" totalImagesPullingTime="3.857529813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:29.089119746 +0000 UTC m=+76.875487756" watchObservedRunningTime="2026-09-15 20:59:29.091669249 +0000 UTC m=+76.878037258"
Sep 15 20:59:29.106885 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.106838    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zzwxd" podStartSLOduration=17.550554627 podStartE2EDuration="21.106821888s" totalImagesPullingTime="3.556267261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:59:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:29.1052772 +0000 UTC m=+76.891645210" watchObservedRunningTime="2026-09-15 20:59:29.106821888 +0000 UTC m=+76.893189895"
Sep 15 20:59:29.169681 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:29.169623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 20:59:29.172655 ip-10-0-131-2 kubenswrapper[2577]: W0915 20:59:29.172626    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8ab60e_3e6b_4dc6_ad96_9cf0abb78cee.slice/crio-1caf52aa167d1a25e8d982a09d50c351d83ec988104e63fd1161b232bcef431e WatchSource:0}: Error finding container 1caf52aa167d1a25e8d982a09d50c351d83ec988104e63fd1161b232bcef431e: Status 404 returned error can't find the container with id 1caf52aa167d1a25e8d982a09d50c351d83ec988104e63fd1161b232bcef431e
Sep 15 20:59:30.083334 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:30.083237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5479955bc7-87qv2" event={"ID":"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee","Type":"ContainerStarted","Data":"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"}
Sep 15 20:59:30.083334 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:30.083328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5479955bc7-87qv2" event={"ID":"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee","Type":"ContainerStarted","Data":"1caf52aa167d1a25e8d982a09d50c351d83ec988104e63fd1161b232bcef431e"}
Sep 15 20:59:30.109288 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:30.109237    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5479955bc7-87qv2" podStartSLOduration=2.109220787 podStartE2EDuration="2.109220787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:30.108214189 +0000 UTC m=+77.894582209" watchObservedRunningTime="2026-09-15 20:59:30.109220787 +0000 UTC m=+77.895588788"
Sep 15 20:59:31.090705 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"aa29dea55910e4e14d69dbd5edc1456304fa954a2c7aa7eb4bcc50b414fa7628"}
Sep 15 20:59:31.091055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"96f32e17192f2fd0c3554bf2b2dc9ca633fdf02254341d9a1766693c51b65139"}
Sep 15 20:59:31.091055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"782ae8d9ee402010b4c6ed5f5c854e2dc7e7e2cd199ec3b2a6e9472b395df6a4"}
Sep 15 20:59:31.091055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"690d7b0c5c74b3004df18562a88d5d13926ac696737c196cc1bd60ff3c58a32b"}
Sep 15 20:59:31.091055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"ea066fe63ce38fc06e575a9d27210bd5dbb9aaa3e1199d0cef107f85e16d2feb"}
Sep 15 20:59:31.091055 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.090845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19f9e565-a82b-4e3c-b9a8-57acb8a9757f","Type":"ContainerStarted","Data":"cbf834845496e69ad90b26d2439728573046e5eeac3f99c85b05994724f4b209"}
Sep 15 20:59:31.118896 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:31.118833    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.622909072 podStartE2EDuration="8.118814866s" totalImagesPullingTime="5.475308215s" totalInitContainerRuntime="20.597579ms" isStatefulPod=false podCreationTimestamp="2026-09-15 20:59:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:31.116707047 +0000 UTC m=+78.903075066" watchObservedRunningTime="2026-09-15 20:59:31.118814866 +0000 UTC m=+78.905182875"
Sep 15 20:59:33.644874 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:33.644831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:59:35.090220 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:35.090193    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d59576f75-txgmp"
Sep 15 20:59:37.205676 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:37.205625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:38.888131 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:38.888096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:38.888550 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:38.888243    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:38.892838 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:38.892817    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:39.125478 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:39.125448    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 20:59:39.169721 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:39.169637    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 20:59:49.094203 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.094138    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-667bf55f74-b7hjz" podUID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" containerName="console" containerID="cri-o://220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561" gracePeriod=15
Sep 15 20:59:49.333523 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.333500    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:49.438881 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.438799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb987\" (UniqueName: \"kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.438881 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.438848    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.439060 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.438908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.439060 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.438933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.439060 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.438971    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.439060 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.439006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert\") pod \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\" (UID: \"ec05cfdb-648a-432e-b0d9-1c97cb50bc68\") "
Sep 15 20:59:49.439370 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.439337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:49.439493 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.439339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config" (OuterVolumeSpecName: "console-config") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:49.439493 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.439375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:49.441419 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.441393    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:49.441503 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.441472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987" (OuterVolumeSpecName: "kube-api-access-gb987") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "kube-api-access-gb987". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:49.441543 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.441498    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec05cfdb-648a-432e-b0d9-1c97cb50bc68" (UID: "ec05cfdb-648a-432e-b0d9-1c97cb50bc68"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:49.540249 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540202    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb987\" (UniqueName: \"kubernetes.io/projected/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-kube-api-access-gb987\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:49.540249 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540242    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:49.540249 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540255    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-oauth-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:49.540249 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540263    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:49.540502 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540271    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-oauth-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:49.540502 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:49.540279    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec05cfdb-648a-432e-b0d9-1c97cb50bc68-console-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:50.167084 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.167047    2577 generic.go:356] "Generic (PLEG): container finished" podID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" containerID="220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561" exitCode=0
Sep 15 20:59:50.167472 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.167099    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-667bf55f74-b7hjz"
Sep 15 20:59:50.167472 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.167135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-667bf55f74-b7hjz" event={"ID":"ec05cfdb-648a-432e-b0d9-1c97cb50bc68","Type":"ContainerDied","Data":"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"}
Sep 15 20:59:50.167472 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.167193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-667bf55f74-b7hjz" event={"ID":"ec05cfdb-648a-432e-b0d9-1c97cb50bc68","Type":"ContainerDied","Data":"acaab16e6ce9c065fd11c051f0858356fc7cbfb644df4ee643891a4c848559d2"}
Sep 15 20:59:50.167472 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.167214    2577 scope.go:117] "RemoveContainer" containerID="220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"
Sep 15 20:59:50.176220 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.176198    2577 scope.go:117] "RemoveContainer" containerID="220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"
Sep 15 20:59:50.176512 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:50.176490    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561\": container with ID starting with 220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561 not found: ID does not exist" containerID="220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"
Sep 15 20:59:50.176612 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.176525    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561"} err="failed to get container status \"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561\": rpc error: code = NotFound desc = could not find container \"220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561\": container with ID starting with 220509f68bc14293e1a3065070f3405f2e64dbb94fac1c1db68b5256f73ec561 not found: ID does not exist"
Sep 15 20:59:50.187233 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.187210    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:50.189974 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.189951    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-667bf55f74-b7hjz"]
Sep 15 20:59:50.689235 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:50.689193    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" path="/var/lib/kubelet/pods/ec05cfdb-648a-432e-b0d9-1c97cb50bc68/volumes"
Sep 15 20:59:55.103660 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.103619    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b5976bc54-zb897" podUID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" containerName="console" containerID="cri-o://bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82" gracePeriod=15
Sep 15 20:59:55.375027 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.374964    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:55.383499 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383482    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383631 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383631 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq2mc\" (UniqueName: \"kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383631 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383625    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383797 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383797 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383682    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383797 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config\") pod \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\" (UID: \"80701e5a-aff3-4336-aecb-4d35dbcf6b31\") "
Sep 15 20:59:55.383989 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.383955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:55.384077 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.384052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca" (OuterVolumeSpecName: "service-ca") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:55.384275 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.384253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config" (OuterVolumeSpecName: "console-config") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:55.384275 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.384258    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:55.385962 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.385942    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:55.386244 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.386227    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc" (OuterVolumeSpecName: "kube-api-access-jq2mc") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "kube-api-access-jq2mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:55.386291 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.386237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "80701e5a-aff3-4336-aecb-4d35dbcf6b31" (UID: "80701e5a-aff3-4336-aecb-4d35dbcf6b31"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:55.484272 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484234    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484272 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484267    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484272 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484278    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-oauth-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484480 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484286    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484480 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484295    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80701e5a-aff3-4336-aecb-4d35dbcf6b31-trusted-ca-bundle\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484480 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484303    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80701e5a-aff3-4336-aecb-4d35dbcf6b31-console-oauth-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:55.484480 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:55.484310    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2mc\" (UniqueName: \"kubernetes.io/projected/80701e5a-aff3-4336-aecb-4d35dbcf6b31-kube-api-access-jq2mc\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 20:59:56.192823 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.192789    2577 generic.go:356] "Generic (PLEG): container finished" podID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" containerID="bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82" exitCode=0
Sep 15 20:59:56.193282 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.193012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5976bc54-zb897" event={"ID":"80701e5a-aff3-4336-aecb-4d35dbcf6b31","Type":"ContainerDied","Data":"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"}
Sep 15 20:59:56.193282 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.193206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5976bc54-zb897" event={"ID":"80701e5a-aff3-4336-aecb-4d35dbcf6b31","Type":"ContainerDied","Data":"8bb66b64e3f25fa49ee48b4ca7e2957e54c359c7657dedba0fbdc709455d080f"}
Sep 15 20:59:56.193282 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.193222    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5976bc54-zb897"
Sep 15 20:59:56.193282 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.193254    2577 scope.go:117] "RemoveContainer" containerID="bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"
Sep 15 20:59:56.204898 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.204882    2577 scope.go:117] "RemoveContainer" containerID="bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"
Sep 15 20:59:56.205145 ip-10-0-131-2 kubenswrapper[2577]: E0915 20:59:56.205129    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82\": container with ID starting with bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82 not found: ID does not exist" containerID="bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"
Sep 15 20:59:56.205190 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.205154    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82"} err="failed to get container status \"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82\": rpc error: code = NotFound desc = could not find container \"bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82\": container with ID starting with bd7bec4b9b029af87448034a33ee3a6090f01cd4c0c6760f7aa4a72375dd9a82 not found: ID does not exist"
Sep 15 20:59:56.216749 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.216725    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:56.220646 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.220622    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b5976bc54-zb897"]
Sep 15 20:59:56.688911 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:56.688879    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" path="/var/lib/kubelet/pods/80701e5a-aff3-4336-aecb-4d35dbcf6b31/volumes"
Sep 15 20:59:57.047952 ip-10-0-131-2 kubenswrapper[2577]: I0915 20:59:57.047923    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9hj9w"
Sep 15 21:00:04.191793 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.191743    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-578f8f844-rzd2v" podUID="e1985a41-b4c3-4177-af1b-1f1e4208982c" containerName="console" containerID="cri-o://8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e" gracePeriod=15
Sep 15 21:00:04.434434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.434397    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 21:00:04.442791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.442699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.442791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.442736    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.442791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.442756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.443072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.442948    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.443072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.442990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpdww\" (UniqueName: \"kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.443072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.443237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config\") pod \"e1985a41-b4c3-4177-af1b-1f1e4208982c\" (UID: \"e1985a41-b4c3-4177-af1b-1f1e4208982c\") "
Sep 15 21:00:04.443293 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443245    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:00:04.443293 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:00:04.443413 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443317    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-oauth-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.443413 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443377    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca" (OuterVolumeSpecName: "service-ca") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:00:04.443556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.443533    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config" (OuterVolumeSpecName: "console-config") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:00:04.445362 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.445325    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:00:04.445483 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.445410    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:00:04.445619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.445591    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww" (OuterVolumeSpecName: "kube-api-access-zpdww") pod "e1985a41-b4c3-4177-af1b-1f1e4208982c" (UID: "e1985a41-b4c3-4177-af1b-1f1e4208982c"). InnerVolumeSpecName "kube-api-access-zpdww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:00:04.544086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544031    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.544086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544081    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpdww\" (UniqueName: \"kubernetes.io/projected/e1985a41-b4c3-4177-af1b-1f1e4208982c-kube-api-access-zpdww\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.544086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544091    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-oauth-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.544086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544100    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.544086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544108    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1985a41-b4c3-4177-af1b-1f1e4208982c-console-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:04.544392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:04.544115    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1985a41-b4c3-4177-af1b-1f1e4208982c-trusted-ca-bundle\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:00:05.229703 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.229668    2577 generic.go:356] "Generic (PLEG): container finished" podID="e1985a41-b4c3-4177-af1b-1f1e4208982c" containerID="8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e" exitCode=0
Sep 15 21:00:05.230096 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.229715    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f8f844-rzd2v"
Sep 15 21:00:05.230096 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.229721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f8f844-rzd2v" event={"ID":"e1985a41-b4c3-4177-af1b-1f1e4208982c","Type":"ContainerDied","Data":"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"}
Sep 15 21:00:05.230096 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.229817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f8f844-rzd2v" event={"ID":"e1985a41-b4c3-4177-af1b-1f1e4208982c","Type":"ContainerDied","Data":"e1f6b73aaff084a6e426af6ffb1a27dd37bc1160635e0451d5e1506774d3b068"}
Sep 15 21:00:05.230096 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.229837    2577 scope.go:117] "RemoveContainer" containerID="8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"
Sep 15 21:00:05.238301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.238285    2577 scope.go:117] "RemoveContainer" containerID="8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"
Sep 15 21:00:05.238660 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:00:05.238639    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e\": container with ID starting with 8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e not found: ID does not exist" containerID="8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"
Sep 15 21:00:05.238766 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.238670    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e"} err="failed to get container status \"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e\": rpc error: code = NotFound desc = could not find container \"8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e\": container with ID starting with 8f5b5fd737f4cb780f317be9c86a8710773d0f1e6c7a00033cd3b2b865138d4e not found: ID does not exist"
Sep 15 21:00:05.246868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.246842    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 21:00:05.250866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:05.250843    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-578f8f844-rzd2v"]
Sep 15 21:00:06.688660 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:06.688622    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1985a41-b4c3-4177-af1b-1f1e4208982c" path="/var/lib/kubelet/pods/e1985a41-b4c3-4177-af1b-1f1e4208982c/volumes"
Sep 15 21:00:23.644966 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:23.644920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 21:00:23.664993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:23.664967    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 21:00:24.316981 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:24.316945    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 21:00:53.383762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.383721    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384135    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384154    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384174    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1985a41-b4c3-4177-af1b-1f1e4208982c" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384181    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1985a41-b4c3-4177-af1b-1f1e4208982c" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384193    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384201    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384286    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1985a41-b4c3-4177-af1b-1f1e4208982c" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384298    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec05cfdb-648a-432e-b0d9-1c97cb50bc68" containerName="console"
Sep 15 21:00:53.384372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.384308    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="80701e5a-aff3-4336-aecb-4d35dbcf6b31" containerName="console"
Sep 15 21:00:53.386204 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.386183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.398395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.398370    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:00:53.532928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.532892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.532943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.533038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.533069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.533091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.533107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjrbw\" (UniqueName: \"kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.533271 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.533183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.633929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.633864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjrbw\" (UniqueName: \"kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.634719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.634694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.634719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.634715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.634896 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.634723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.634896 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.634844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.636401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.636371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.636474 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.636437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.643424 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.643403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjrbw\" (UniqueName: \"kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw\") pod \"console-854d856c-9v9px\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") " pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.695203 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.695167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:00:53.965697 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:53.965621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:00:53.968354 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:00:53.968322    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8d35fa_ef7a_49b7_aa45_6bb947a2c5f7.slice/crio-d15c72e98d61b7b4baf9fd3c652d67f07fa047332e6d4e669fa48b56c1abede0 WatchSource:0}: Error finding container d15c72e98d61b7b4baf9fd3c652d67f07fa047332e6d4e669fa48b56c1abede0: Status 404 returned error can't find the container with id d15c72e98d61b7b4baf9fd3c652d67f07fa047332e6d4e669fa48b56c1abede0
Sep 15 21:00:54.412103 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:54.412015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-854d856c-9v9px" event={"ID":"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7","Type":"ContainerStarted","Data":"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"}
Sep 15 21:00:54.412103 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:54.412130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-854d856c-9v9px" event={"ID":"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7","Type":"ContainerStarted","Data":"d15c72e98d61b7b4baf9fd3c652d67f07fa047332e6d4e669fa48b56c1abede0"}
Sep 15 21:00:54.435772 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:00:54.435723    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-854d856c-9v9px" podStartSLOduration=1.435704087 podStartE2EDuration="1.435704087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:00:54.434540286 +0000 UTC m=+162.220908310" watchObservedRunningTime="2026-09-15 21:00:54.435704087 +0000 UTC m=+162.222072094"
Sep 15 21:01:03.696342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:03.696303    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:01:03.696842 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:03.696360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:01:03.701033 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:03.701011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:01:04.452350 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:04.452324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:01:04.501023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:04.500993    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 21:01:29.520715 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.520673    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5479955bc7-87qv2" podUID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" containerName="console" containerID="cri-o://0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02" gracePeriod=15
Sep 15 21:01:29.759360 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.759332    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 21:01:29.814321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814246    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814324    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bdhp\" (UniqueName: \"kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814363    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814425    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca\") pod \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\" (UID: \"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee\") "
Sep 15 21:01:29.814788 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814714    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:01:29.814928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config" (OuterVolumeSpecName: "console-config") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:01:29.814977 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814920    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca" (OuterVolumeSpecName: "service-ca") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:01:29.814977 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.814901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:01:29.816659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.816633    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:01:29.816659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.816651    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:01:29.816811 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.816796    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp" (OuterVolumeSpecName: "kube-api-access-4bdhp") pod "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" (UID: "1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee"). InnerVolumeSpecName "kube-api-access-4bdhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:01:29.915472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915440    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915492    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-oauth-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915502    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915510    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-oauth-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915518    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bdhp\" (UniqueName: \"kubernetes.io/projected/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-kube-api-access-4bdhp\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915526    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-trusted-ca-bundle\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:29.915661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:29.915534    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee-console-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:01:30.552745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.552707    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" containerID="0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02" exitCode=0
Sep 15 21:01:30.553146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.552756    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5479955bc7-87qv2"
Sep 15 21:01:30.553146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.552790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5479955bc7-87qv2" event={"ID":"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee","Type":"ContainerDied","Data":"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"}
Sep 15 21:01:30.553146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.552839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5479955bc7-87qv2" event={"ID":"1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee","Type":"ContainerDied","Data":"1caf52aa167d1a25e8d982a09d50c351d83ec988104e63fd1161b232bcef431e"}
Sep 15 21:01:30.553146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.552860    2577 scope.go:117] "RemoveContainer" containerID="0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"
Sep 15 21:01:30.561658 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.561643    2577 scope.go:117] "RemoveContainer" containerID="0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"
Sep 15 21:01:30.561967 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:01:30.561951    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02\": container with ID starting with 0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02 not found: ID does not exist" containerID="0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"
Sep 15 21:01:30.562019 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.561987    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02"} err="failed to get container status \"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02\": rpc error: code = NotFound desc = could not find container \"0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02\": container with ID starting with 0db4f98da21be3c107788eb554472e7ac287ff0c7e2c6feb7b242f0c7801ff02 not found: ID does not exist"
Sep 15 21:01:30.574537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.574511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 21:01:30.578453 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.578426    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5479955bc7-87qv2"]
Sep 15 21:01:30.688845 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:01:30.688815    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" path="/var/lib/kubelet/pods/1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee/volumes"
Sep 15 21:02:02.660694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.659425    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xghb5"]
Sep 15 21:02:02.660694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.659854    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" containerName="console"
Sep 15 21:02:02.660694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.659871    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" containerName="console"
Sep 15 21:02:02.660694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.659943    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d8ab60e-3e6b-4dc6-ad96-9cf0abb78cee" containerName="console"
Sep 15 21:02:02.662930 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.662911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.666008 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.665987    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 15 21:02:02.666452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.666433    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 15 21:02:02.667373 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.667349    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-tvzc2\""
Sep 15 21:02:02.676880 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.676860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xghb5"]
Sep 15 21:02:02.782269 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.782227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-kube-api-access-qh9cz\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.782521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.782495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-bound-sa-token\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.883549 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.883513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-kube-api-access-qh9cz\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.883709 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.883601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-bound-sa-token\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.891780 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.891745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-kube-api-access-qh9cz\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.891881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.891797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0dd9354-7bff-4c1c-b880-7d70624d18f3-bound-sa-token\") pod \"cert-manager-55b4d954d5-xghb5\" (UID: \"a0dd9354-7bff-4c1c-b880-7d70624d18f3\") " pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:02.982675 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:02.982653    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xghb5"
Sep 15 21:02:03.273608 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:03.273580    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xghb5"]
Sep 15 21:02:03.275867 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:02:03.275842    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0dd9354_7bff_4c1c_b880_7d70624d18f3.slice/crio-4ce33e4e0cce9bb097c9a56a78c02dd1552b2fc5d0456981f7f26d73a5ae107f WatchSource:0}: Error finding container 4ce33e4e0cce9bb097c9a56a78c02dd1552b2fc5d0456981f7f26d73a5ae107f: Status 404 returned error can't find the container with id 4ce33e4e0cce9bb097c9a56a78c02dd1552b2fc5d0456981f7f26d73a5ae107f
Sep 15 21:02:03.674970 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:03.674886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xghb5" event={"ID":"a0dd9354-7bff-4c1c-b880-7d70624d18f3","Type":"ContainerStarted","Data":"4ce33e4e0cce9bb097c9a56a78c02dd1552b2fc5d0456981f7f26d73a5ae107f"}
Sep 15 21:02:06.692335 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:06.692260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xghb5" event={"ID":"a0dd9354-7bff-4c1c-b880-7d70624d18f3","Type":"ContainerStarted","Data":"05d063036228448350781eb133c68d25050e312106794748ba5f4e7fe4c5fb55"}
Sep 15 21:02:06.713445 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:06.713397    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xghb5" podStartSLOduration=1.5500639440000001 podStartE2EDuration="4.713380182s" totalImagesPullingTime="3.163316238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:02:06.712507493 +0000 UTC m=+234.498875502" watchObservedRunningTime="2026-09-15 21:02:06.713380182 +0000 UTC m=+234.499748190"
Sep 15 21:02:31.479095 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.479062    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"]
Sep 15 21:02:31.485178 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.485157    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.487656 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.487629    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 21:02:31.487925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.487907    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 21:02:31.488034 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.487923    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wj45x\""
Sep 15 21:02:31.488034 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.487921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 21:02:31.488159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.488142    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 21:02:31.501749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.501726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"]
Sep 15 21:02:31.512972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.512948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxlmq\" (UniqueName: \"kubernetes.io/projected/86152b83-66b7-4871-b361-3bdec8959af9-kube-api-access-pxlmq\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.513110 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.513020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.513110 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.513096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.513213 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.513130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.513213 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.513198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614337 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxlmq\" (UniqueName: \"kubernetes.io/projected/86152b83-66b7-4871-b361-3bdec8959af9-kube-api-access-pxlmq\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.614849 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.614829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.616916 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.616889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86152b83-66b7-4871-b361-3bdec8959af9-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.617024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.616945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.617024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.616972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86152b83-66b7-4871-b361-3bdec8959af9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.631610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.631588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxlmq\" (UniqueName: \"kubernetes.io/projected/86152b83-66b7-4871-b361-3bdec8959af9-kube-api-access-pxlmq\") pod \"opendatahub-operator-controller-manager-6f85789df9-wzdh8\" (UID: \"86152b83-66b7-4871-b361-3bdec8959af9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:31.795699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:31.795618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:32.082005 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:32.081979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"]
Sep 15 21:02:32.084634 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:02:32.084609    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86152b83_66b7_4871_b361_3bdec8959af9.slice/crio-701b1b29bc1ac3833f0c39984cdb5a81f7786cdb13309a14867519784f77638b WatchSource:0}: Error finding container 701b1b29bc1ac3833f0c39984cdb5a81f7786cdb13309a14867519784f77638b: Status 404 returned error can't find the container with id 701b1b29bc1ac3833f0c39984cdb5a81f7786cdb13309a14867519784f77638b
Sep 15 21:02:32.788497 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:32.788454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8" event={"ID":"86152b83-66b7-4871-b361-3bdec8959af9","Type":"ContainerStarted","Data":"701b1b29bc1ac3833f0c39984cdb5a81f7786cdb13309a14867519784f77638b"}
Sep 15 21:02:35.803881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:35.803849    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8" event={"ID":"86152b83-66b7-4871-b361-3bdec8959af9","Type":"ContainerStarted","Data":"050d648b97968c0a6fe263def627d1b2ad326821b0286ee791213e39233aef9f"}
Sep 15 21:02:36.808790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:36.808708    2577 generic.go:356] "Generic (PLEG): container finished" podID="86152b83-66b7-4871-b361-3bdec8959af9" containerID="050d648b97968c0a6fe263def627d1b2ad326821b0286ee791213e39233aef9f" exitCode=0
Sep 15 21:02:36.809126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:36.808793    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8" event={"ID":"86152b83-66b7-4871-b361-3bdec8959af9","Type":"ContainerDied","Data":"050d648b97968c0a6fe263def627d1b2ad326821b0286ee791213e39233aef9f"}
Sep 15 21:02:37.815024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:37.814989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8" event={"ID":"86152b83-66b7-4871-b361-3bdec8959af9","Type":"ContainerStarted","Data":"4125ab46fa41321ceab1f8bffd2e3b986e286428e8a8369e4075d1869e48ae7e"}
Sep 15 21:02:37.815429 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:37.815068    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:37.839467 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:37.839420    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8" podStartSLOduration=2.143376776 podStartE2EDuration="6.839404559s" totalImagesPullingTime="3.676700725s" totalInitContainerRuntime="1.019327058s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:02:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:02:37.838116217 +0000 UTC m=+265.624484225" watchObservedRunningTime="2026-09-15 21:02:37.839404559 +0000 UTC m=+265.625772567"
Sep 15 21:02:48.821213 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:48.821180    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-wzdh8"
Sep 15 21:02:51.350833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.350788    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"]
Sep 15 21:02:51.355952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.355934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.358543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.358520    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 21:02:51.359083 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.359071    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 21:02:51.359257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.359239    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 21:02:51.359808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.359794    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 21:02:51.373290 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.373267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"]
Sep 15 21:02:51.495175 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.495144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8j98\" (UniqueName: \"kubernetes.io/projected/4eb76639-bc49-4805-a0e4-7e27af5a8525-kube-api-access-t8j98\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.495345 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.495195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb76639-bc49-4805-a0e4-7e27af5a8525-tmp\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.495345 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.495261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4eb76639-bc49-4805-a0e4-7e27af5a8525-tls-certs\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.596671 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.596633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8j98\" (UniqueName: \"kubernetes.io/projected/4eb76639-bc49-4805-a0e4-7e27af5a8525-kube-api-access-t8j98\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.596866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.596704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb76639-bc49-4805-a0e4-7e27af5a8525-tmp\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.596866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.596753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4eb76639-bc49-4805-a0e4-7e27af5a8525-tls-certs\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.599190 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.599164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb76639-bc49-4805-a0e4-7e27af5a8525-tmp\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.599404 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.599384    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4eb76639-bc49-4805-a0e4-7e27af5a8525-tls-certs\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.614406 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.614343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8j98\" (UniqueName: \"kubernetes.io/projected/4eb76639-bc49-4805-a0e4-7e27af5a8525-kube-api-access-t8j98\") pod \"kube-auth-proxy-85f7c7997-x5n9t\" (UID: \"4eb76639-bc49-4805-a0e4-7e27af5a8525\") " pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.666156 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.666126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"
Sep 15 21:02:51.935871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:51.935843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t"]
Sep 15 21:02:51.938341 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:02:51.938314    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb76639_bc49_4805_a0e4_7e27af5a8525.slice/crio-11991d26fdb31d0a1c830b965e415ac540afd9c2b8ae4f59b2ede52e745ddf66 WatchSource:0}: Error finding container 11991d26fdb31d0a1c830b965e415ac540afd9c2b8ae4f59b2ede52e745ddf66: Status 404 returned error can't find the container with id 11991d26fdb31d0a1c830b965e415ac540afd9c2b8ae4f59b2ede52e745ddf66
Sep 15 21:02:52.887100 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:52.887025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t" event={"ID":"4eb76639-bc49-4805-a0e4-7e27af5a8525","Type":"ContainerStarted","Data":"11991d26fdb31d0a1c830b965e415ac540afd9c2b8ae4f59b2ede52e745ddf66"}
Sep 15 21:02:54.896810 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:54.896726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t" event={"ID":"4eb76639-bc49-4805-a0e4-7e27af5a8525","Type":"ContainerStarted","Data":"4ce62b2cacff2d9154eacad710a0a33bc142ccba1e991ce17e8d8937f473776e"}
Sep 15 21:02:54.914254 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:54.914202    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85f7c7997-x5n9t" podStartSLOduration=1.245971786 podStartE2EDuration="3.914188612s" totalImagesPullingTime="2.668216826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:02:54.91302196 +0000 UTC m=+282.699389968" watchObservedRunningTime="2026-09-15 21:02:54.914188612 +0000 UTC m=+282.700556621"
Sep 15 21:02:58.844469 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.844434    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"]
Sep 15 21:02:58.846896 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.846875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.849664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.849630    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 21:02:58.849807 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.849759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bcmg8\""
Sep 15 21:02:58.861595 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.861555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"]
Sep 15 21:02:58.966206 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9316c44-b627-454b-ad8a-e1c19d2687fa-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bxfk\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-kube-api-access-9bxfk\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:58.966536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:58.966492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.067846 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9316c44-b627-454b-ad8a-e1c19d2687fa-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.067846 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.067981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bxfk\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-kube-api-access-9bxfk\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068451 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068451 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068588 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068759 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.068933 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.068916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9316c44-b627-454b-ad8a-e1c19d2687fa-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.070480 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.070462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.071243 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.071219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.075785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.075764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.075961 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.075944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bxfk\" (UniqueName: \"kubernetes.io/projected/f9316c44-b627-454b-ad8a-e1c19d2687fa-kube-api-access-9bxfk\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5\" (UID: \"f9316c44-b627-454b-ad8a-e1c19d2687fa\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.156979 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.156888    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:02:59.428986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.428894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"]
Sep 15 21:02:59.431559 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:02:59.431527    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9316c44_b627_454b_ad8a_e1c19d2687fa.slice/crio-387d62415a1e35c5be6526076ba56af5da31c35e6f5f6bf491acff499007b84d WatchSource:0}: Error finding container 387d62415a1e35c5be6526076ba56af5da31c35e6f5f6bf491acff499007b84d: Status 404 returned error can't find the container with id 387d62415a1e35c5be6526076ba56af5da31c35e6f5f6bf491acff499007b84d
Sep 15 21:02:59.917247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:02:59.917210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5" event={"ID":"f9316c44-b627-454b-ad8a-e1c19d2687fa","Type":"ContainerStarted","Data":"387d62415a1e35c5be6526076ba56af5da31c35e6f5f6bf491acff499007b84d"}
Sep 15 21:03:02.137792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.137756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-246nq"]
Sep 15 21:03:02.145081 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.145036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.148274 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.148216    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 15 21:03:02.148734 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.148714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-x6n2s\""
Sep 15 21:03:02.153219 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.153194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-246nq"]
Sep 15 21:03:02.189897 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.189840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmm5z\" (UniqueName: \"kubernetes.io/projected/489116ed-5fa3-4132-b563-b834a175f16e-kube-api-access-xmm5z\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.190069 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.189904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.291462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.291395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmm5z\" (UniqueName: \"kubernetes.io/projected/489116ed-5fa3-4132-b563-b834a175f16e-kube-api-access-xmm5z\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.292454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.291735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.292454 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:02.291932    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 15 21:03:02.292454 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:02.292009    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert podName:489116ed-5fa3-4132-b563-b834a175f16e nodeName:}" failed. No retries permitted until 2026-09-15 21:03:02.791986556 +0000 UTC m=+290.578354555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert") pod "odh-model-controller-7998c67dc4-246nq" (UID: "489116ed-5fa3-4132-b563-b834a175f16e") : secret "odh-model-controller-webhook-cert" not found
Sep 15 21:03:02.301224 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.301193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmm5z\" (UniqueName: \"kubernetes.io/projected/489116ed-5fa3-4132-b563-b834a175f16e-kube-api-access-xmm5z\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.487435 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.487395    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:03:02.487544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.487484    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:03:02.487544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.487526    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:03:02.796995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.796902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.799637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.799604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/489116ed-5fa3-4132-b563-b834a175f16e-cert\") pod \"odh-model-controller-7998c67dc4-246nq\" (UID: \"489116ed-5fa3-4132-b563-b834a175f16e\") " pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:02.803172 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.803146    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sfsgb"]
Sep 15 21:03:02.805944 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.805927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:02.808374 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.808356    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 15 21:03:02.808469 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.808394    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bvvqv\""
Sep 15 21:03:02.814292 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.814272    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sfsgb"]
Sep 15 21:03:02.897517 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.897482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:02.897741 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.897532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh99n\" (UniqueName: \"kubernetes.io/projected/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-kube-api-access-kh99n\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:02.931958 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.931921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5" event={"ID":"f9316c44-b627-454b-ad8a-e1c19d2687fa","Type":"ContainerStarted","Data":"ad1a0bf8770a2b782a19e622f54f4b136c6c6056a8296c9cbb620037542f93e7"}
Sep 15 21:03:02.954131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.954076    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5" podStartSLOduration=1.9004725630000001 podStartE2EDuration="4.954053184s" totalImagesPullingTime="3.053580621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:02.952454477 +0000 UTC m=+290.738822486" watchObservedRunningTime="2026-09-15 21:03:02.954053184 +0000 UTC m=+290.740421191"
Sep 15 21:03:02.998397 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.998372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:02.998540 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:02.998417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh99n\" (UniqueName: \"kubernetes.io/projected/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-kube-api-access-kh99n\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:02.998540 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:02.998510    2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 15 21:03:02.998680 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:02.998608    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs podName:05ef6d78-cfbd-4ce0-94c2-465be0e159c6 nodeName:}" failed. No retries permitted until 2026-09-15 21:03:03.498556497 +0000 UTC m=+291.284924502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs") pod "model-serving-api-54d89bd5fc-sfsgb" (UID: "05ef6d78-cfbd-4ce0-94c2-465be0e159c6") : secret "model-serving-api-tls" not found
Sep 15 21:03:03.010116 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.010091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh99n\" (UniqueName: \"kubernetes.io/projected/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-kube-api-access-kh99n\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:03.065317 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.065221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:03.157666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.157633    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:03:03.162723 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.162691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:03:03.338227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.338137    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-246nq"]
Sep 15 21:03:03.341688 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:03:03.341637    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489116ed_5fa3_4132_b563_b834a175f16e.slice/crio-c7468761e26a77190d48bd2f831191dbd21ca9840eccc910486766be8bd868fa WatchSource:0}: Error finding container c7468761e26a77190d48bd2f831191dbd21ca9840eccc910486766be8bd868fa: Status 404 returned error can't find the container with id c7468761e26a77190d48bd2f831191dbd21ca9840eccc910486766be8bd868fa
Sep 15 21:03:03.502017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.501982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:03.504594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.504537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ef6d78-cfbd-4ce0-94c2-465be0e159c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-sfsgb\" (UID: \"05ef6d78-cfbd-4ce0-94c2-465be0e159c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:03.716965 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.716879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:03.938878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.938842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-246nq" event={"ID":"489116ed-5fa3-4132-b563-b834a175f16e","Type":"ContainerStarted","Data":"c7468761e26a77190d48bd2f831191dbd21ca9840eccc910486766be8bd868fa"}
Sep 15 21:03:03.939172 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.939154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:03:03.940144 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.940123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5"
Sep 15 21:03:03.995717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:03.995557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sfsgb"]
Sep 15 21:03:03.997829 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:03:03.997795    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ef6d78_cfbd_4ce0_94c2_465be0e159c6.slice/crio-bfa6845d6a863efd1bbe897af805c1553f442e9a38bcb271dbe1902886c4e0eb WatchSource:0}: Error finding container bfa6845d6a863efd1bbe897af805c1553f442e9a38bcb271dbe1902886c4e0eb: Status 404 returned error can't find the container with id bfa6845d6a863efd1bbe897af805c1553f442e9a38bcb271dbe1902886c4e0eb
Sep 15 21:03:04.946641 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:04.946602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb" event={"ID":"05ef6d78-cfbd-4ce0-94c2-465be0e159c6","Type":"ContainerStarted","Data":"bfa6845d6a863efd1bbe897af805c1553f442e9a38bcb271dbe1902886c4e0eb"}
Sep 15 21:03:07.965869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:07.965829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-246nq" event={"ID":"489116ed-5fa3-4132-b563-b834a175f16e","Type":"ContainerStarted","Data":"c5db91d5498f7f5aed89bf7131fc503e57b0bca2a1a5220a2fb07eda1addbd1b"}
Sep 15 21:03:07.966340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:07.965921    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:07.967988 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:07.967953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb" event={"ID":"05ef6d78-cfbd-4ce0-94c2-465be0e159c6","Type":"ContainerStarted","Data":"fc89296312ba9c1018a771b419e1fb1c2505064d604493ac583a727e9f0768a7"}
Sep 15 21:03:07.968117 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:07.968007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:07.982672 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:07.982624    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-246nq" podStartSLOduration=2.036937479 podStartE2EDuration="5.982606199s" totalImagesPullingTime="3.94566872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:07.982456027 +0000 UTC m=+295.768824034" watchObservedRunningTime="2026-09-15 21:03:07.982606199 +0000 UTC m=+295.768974206"
Sep 15 21:03:08.001654 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:08.001608    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb" podStartSLOduration=2.721498417 podStartE2EDuration="6.001593029s" totalImagesPullingTime="3.280094612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:07.999983519 +0000 UTC m=+295.786351527" watchObservedRunningTime="2026-09-15 21:03:08.001593029 +0000 UTC m=+295.787961037"
Sep 15 21:03:09.825101 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.825065    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"]
Sep 15 21:03:09.827321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.827304    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.830042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.830012    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 21:03:09.830160 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.830012    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 21:03:09.830160 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.830109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 21:03:09.830248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.830210    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sgljd\""
Sep 15 21:03:09.841062 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.841042    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"]
Sep 15 21:03:09.860109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ef9e7d5-84be-493b-b871-923f429abd99-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mxp9\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-kube-api-access-8mxp9\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.860465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.860315    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961653 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ef9e7d5-84be-493b-b871-923f429abd99-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mxp9\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-kube-api-access-8mxp9\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.961825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.961811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.962809 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.962782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.964263 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.964236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ef9e7d5-84be-493b-b871-923f429abd99-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.964367 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.964343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.964447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.964399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.964447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.964413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0ef9e7d5-84be-493b-b871-923f429abd99-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.969791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.969768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:09.969866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:09.969824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mxp9\" (UniqueName: \"kubernetes.io/projected/0ef9e7d5-84be-493b-b871-923f429abd99-kube-api-access-8mxp9\") pod \"istiod-openshift-gateway-5677ccb49b-nxgmr\" (UID: \"0ef9e7d5-84be-493b-b871-923f429abd99\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:10.136791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:10.136716    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:10.428905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:10.428882    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"]
Sep 15 21:03:10.431649 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:03:10.431606    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef9e7d5_84be_493b_b871_923f429abd99.slice/crio-6f3dcb11ad47c197fd68a40833219d33b0cc422597bee640424ddb33a27189b5 WatchSource:0}: Error finding container 6f3dcb11ad47c197fd68a40833219d33b0cc422597bee640424ddb33a27189b5: Status 404 returned error can't find the container with id 6f3dcb11ad47c197fd68a40833219d33b0cc422597bee640424ddb33a27189b5
Sep 15 21:03:10.983549 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:10.983511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr" event={"ID":"0ef9e7d5-84be-493b-b871-923f429abd99","Type":"ContainerStarted","Data":"6f3dcb11ad47c197fd68a40833219d33b0cc422597bee640424ddb33a27189b5"}
Sep 15 21:03:12.522637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.522603    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"]
Sep 15 21:03:12.524961 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.524940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.528321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.528115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 15 21:03:12.528450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.528356    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-9xb48\""
Sep 15 21:03:12.536389 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.536369    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"]
Sep 15 21:03:12.538980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.538957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:12.540460 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.540442    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"]
Sep 15 21:03:12.542904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.542889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mhnqw\""
Sep 15 21:03:12.543118 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.543102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 21:03:12.560059 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.560030    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"]
Sep 15 21:03:12.586159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.586135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:03:12.589007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.588990    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:03:12.591382 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.591359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwgz5\" (UniqueName: \"kubernetes.io/projected/3962480d-e0db-4836-9890-3ec402d58dee-kube-api-access-dwgz5\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.591500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.591482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:12.591595 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.591527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff8j2\" (UniqueName: \"kubernetes.io/projected/2f654635-5148-406c-8c1f-9e9140abc361-kube-api-access-ff8j2\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:12.591595 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.591589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.595059 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.595038    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 21:03:12.692195 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.692055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwgz5\" (UniqueName: \"kubernetes.io/projected/3962480d-e0db-4836-9890-3ec402d58dee-kube-api-access-dwgz5\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.692195 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.692170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.692210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff8j2\" (UniqueName: \"kubernetes.io/projected/2f654635-5148-406c-8c1f-9e9140abc361-kube-api-access-ff8j2\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.692244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:12.692339    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:12.692352    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:12.692405    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert podName:2f654635-5148-406c-8c1f-9e9140abc361 nodeName:}" failed. No retries permitted until 2026-09-15 21:03:13.192383766 +0000 UTC m=+300.978751766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert") pod "llmisvc-controller-manager-777c6bd7d5-hfcpz" (UID: "2f654635-5148-406c-8c1f-9e9140abc361") : secret "llmisvc-webhook-server-cert" not found
Sep 15 21:03:12.699105 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:03:12.692436    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert podName:3962480d-e0db-4836-9890-3ec402d58dee nodeName:}" failed. No retries permitted until 2026-09-15 21:03:13.192415197 +0000 UTC m=+300.978783184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert") pod "kserve-controller-manager-d6f4b65b8-c6ckd" (UID: "3962480d-e0db-4836-9890-3ec402d58dee") : secret "kserve-webhook-server-cert" not found
Sep 15 21:03:12.715203 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.715177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwgz5\" (UniqueName: \"kubernetes.io/projected/3962480d-e0db-4836-9890-3ec402d58dee-kube-api-access-dwgz5\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:12.715350 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:12.715335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff8j2\" (UniqueName: \"kubernetes.io/projected/2f654635-5148-406c-8c1f-9e9140abc361-kube-api-access-ff8j2\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:13.197529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.197488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:13.197727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.197559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:13.200370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.200348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f654635-5148-406c-8c1f-9e9140abc361-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hfcpz\" (UID: \"2f654635-5148-406c-8c1f-9e9140abc361\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:13.200508 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.200400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3962480d-e0db-4836-9890-3ec402d58dee-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6ckd\" (UID: \"3962480d-e0db-4836-9890-3ec402d58dee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:13.436557 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.436522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:13.450322 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.450257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:13.618822 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.618774    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:03:13.619220 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.618867    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:03:13.860285 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.860254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"]
Sep 15 21:03:13.863715 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.863690    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"]
Sep 15 21:03:13.864255 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:03:13.864227    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f654635_5148_406c_8c1f_9e9140abc361.slice/crio-db1bec4afb98ccd07a24784943ff8c47380f219d8a4fc26d770021ba2cb78a54 WatchSource:0}: Error finding container db1bec4afb98ccd07a24784943ff8c47380f219d8a4fc26d770021ba2cb78a54: Status 404 returned error can't find the container with id db1bec4afb98ccd07a24784943ff8c47380f219d8a4fc26d770021ba2cb78a54
Sep 15 21:03:13.865744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:13.865727    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:03:13.866343 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:03:13.866257    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3962480d_e0db_4836_9890_3ec402d58dee.slice/crio-53466b3b6cc22a00134844d974cc3f0978ec0dfb25fef64b35c84076889233b1 WatchSource:0}: Error finding container 53466b3b6cc22a00134844d974cc3f0978ec0dfb25fef64b35c84076889233b1: Status 404 returned error can't find the container with id 53466b3b6cc22a00134844d974cc3f0978ec0dfb25fef64b35c84076889233b1
Sep 15 21:03:14.000956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.000923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz" event={"ID":"2f654635-5148-406c-8c1f-9e9140abc361","Type":"ContainerStarted","Data":"db1bec4afb98ccd07a24784943ff8c47380f219d8a4fc26d770021ba2cb78a54"}
Sep 15 21:03:14.002629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.002592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr" event={"ID":"0ef9e7d5-84be-493b-b871-923f429abd99","Type":"ContainerStarted","Data":"287b8486257a540a0dc6a57ebb80895769e5d1a9d816c37ee731d3280cba10c9"}
Sep 15 21:03:14.002786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.002767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:14.003862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.003840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd" event={"ID":"3962480d-e0db-4836-9890-3ec402d58dee","Type":"ContainerStarted","Data":"53466b3b6cc22a00134844d974cc3f0978ec0dfb25fef64b35c84076889233b1"}
Sep 15 21:03:14.004618 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.004594    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-nxgmr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 21:03:14.004721 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.004648    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr" podUID="0ef9e7d5-84be-493b-b871-923f429abd99" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 21:03:14.026795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:14.026754    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr" podStartSLOduration=1.8424419570000001 podStartE2EDuration="5.026739181s" totalImagesPullingTime="3.184297224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:14.024615729 +0000 UTC m=+301.810983735" watchObservedRunningTime="2026-09-15 21:03:14.026739181 +0000 UTC m=+301.813107188"
Sep 15 21:03:15.010070 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:15.010043    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nxgmr"
Sep 15 21:03:18.024862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.024824    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz" event={"ID":"2f654635-5148-406c-8c1f-9e9140abc361","Type":"ContainerStarted","Data":"6fb33b96c48dd1399b587614bb97dc19dbf6858cbee7b3e6359d0609198896a2"}
Sep 15 21:03:18.025304 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.024983    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:18.026329 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.026299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd" event={"ID":"3962480d-e0db-4836-9890-3ec402d58dee","Type":"ContainerStarted","Data":"26521755076c9e1f302433748f33ebff27308259bfbd5bec93193b9a71a2760b"}
Sep 15 21:03:18.026433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.026414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:03:18.056408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.056357    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz" podStartSLOduration=2.536186726 podStartE2EDuration="6.056340989s" totalImagesPullingTime="3.520154263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:18.054312231 +0000 UTC m=+305.840680240" watchObservedRunningTime="2026-09-15 21:03:18.056340989 +0000 UTC m=+305.842708997"
Sep 15 21:03:18.081475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.081426    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd" podStartSLOduration=2.56133083 podStartE2EDuration="6.081413713s" totalImagesPullingTime="3.520082883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:18.080452597 +0000 UTC m=+305.866820589" watchObservedRunningTime="2026-09-15 21:03:18.081413713 +0000 UTC m=+305.867781763"
Sep 15 21:03:18.974623 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.974593    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-246nq"
Sep 15 21:03:18.976666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:18.976647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-sfsgb"
Sep 15 21:03:49.033233 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:49.033202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hfcpz"
Sep 15 21:03:49.036541 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:03:49.036516    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6ckd"
Sep 15 21:04:03.905893 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.905815    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68fc8ddc86-6xmbp"]
Sep 15 21:04:03.908113 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.908094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.925384 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.925357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68fc8ddc86-6xmbp"]
Sep 15 21:04:03.954283 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-trusted-ca-bundle\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-oauth-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954624 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-oauth-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954624 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-service-ca\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:03.954624 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:03.954613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjgr\" (UniqueName: \"kubernetes.io/projected/0cf7f666-7840-4ad0-9b4f-35ad16eea799-kube-api-access-cqjgr\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.055904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.055863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-oauth-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056101 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.055916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-service-ca\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056197 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjgr\" (UniqueName: \"kubernetes.io/projected/0cf7f666-7840-4ad0-9b4f-35ad16eea799-kube-api-access-cqjgr\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056274 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-trusted-ca-bundle\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056334 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-oauth-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.056788 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.056766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-service-ca\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.057198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.057163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-trusted-ca-bundle\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.057336 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.057314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.057413 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.057345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf7f666-7840-4ad0-9b4f-35ad16eea799-oauth-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.058950 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.058926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-oauth-config\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.058950 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.058944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf7f666-7840-4ad0-9b4f-35ad16eea799-console-serving-cert\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.065018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.064998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjgr\" (UniqueName: \"kubernetes.io/projected/0cf7f666-7840-4ad0-9b4f-35ad16eea799-kube-api-access-cqjgr\") pod \"console-68fc8ddc86-6xmbp\" (UID: \"0cf7f666-7840-4ad0-9b4f-35ad16eea799\") " pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.217348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.217269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:04.493978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:04.493948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68fc8ddc86-6xmbp"]
Sep 15 21:04:04.497189 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:04.497162    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf7f666_7840_4ad0_9b4f_35ad16eea799.slice/crio-53b4eeba08cae5b3e2565b671d38ad6b7586b9bf55f451b4aa35854ffe3762b8 WatchSource:0}: Error finding container 53b4eeba08cae5b3e2565b671d38ad6b7586b9bf55f451b4aa35854ffe3762b8: Status 404 returned error can't find the container with id 53b4eeba08cae5b3e2565b671d38ad6b7586b9bf55f451b4aa35854ffe3762b8
Sep 15 21:04:05.230773 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:05.230725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68fc8ddc86-6xmbp" event={"ID":"0cf7f666-7840-4ad0-9b4f-35ad16eea799","Type":"ContainerStarted","Data":"f512a93a791a4512adae77b5bceffa0fd804774f08fe904e0fdab5f7ad0ff769"}
Sep 15 21:04:05.231164 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:05.230788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68fc8ddc86-6xmbp" event={"ID":"0cf7f666-7840-4ad0-9b4f-35ad16eea799","Type":"ContainerStarted","Data":"53b4eeba08cae5b3e2565b671d38ad6b7586b9bf55f451b4aa35854ffe3762b8"}
Sep 15 21:04:05.249542 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:05.249493    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68fc8ddc86-6xmbp" podStartSLOduration=2.249477572 podStartE2EDuration="2.249477572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:05.248010011 +0000 UTC m=+353.034378017" watchObservedRunningTime="2026-09-15 21:04:05.249477572 +0000 UTC m=+353.035845580"
Sep 15 21:04:14.217659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:14.217620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:14.218082 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:14.217683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:14.223688 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:14.223651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:14.275785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:14.275761    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68fc8ddc86-6xmbp"
Sep 15 21:04:14.349401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:14.349369    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:04:20.566653 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.566623    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"]
Sep 15 21:04:20.569871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.569846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:20.572754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.572730    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 21:04:20.572754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.572748    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 15 21:04:20.573227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.573205    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cdcqb\""
Sep 15 21:04:20.573928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.573912    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 21:04:20.591771 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.591747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"]
Sep 15 21:04:20.709574 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.709531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cl5x\" (UniqueName: \"kubernetes.io/projected/ea87473d-8756-4cd4-bf5b-fdcc98bd30e6-kube-api-access-8cl5x\") pod \"dns-operator-controller-manager-7b84cd7c9c-6qw9x\" (UID: \"ea87473d-8756-4cd4-bf5b-fdcc98bd30e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:20.810948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.810910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cl5x\" (UniqueName: \"kubernetes.io/projected/ea87473d-8756-4cd4-bf5b-fdcc98bd30e6-kube-api-access-8cl5x\") pod \"dns-operator-controller-manager-7b84cd7c9c-6qw9x\" (UID: \"ea87473d-8756-4cd4-bf5b-fdcc98bd30e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:20.823218 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.823145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cl5x\" (UniqueName: \"kubernetes.io/projected/ea87473d-8756-4cd4-bf5b-fdcc98bd30e6-kube-api-access-8cl5x\") pod \"dns-operator-controller-manager-7b84cd7c9c-6qw9x\" (UID: \"ea87473d-8756-4cd4-bf5b-fdcc98bd30e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:20.881456 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:20.881415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:21.156204 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:21.156179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"]
Sep 15 21:04:21.158880 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:21.158845    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea87473d_8756_4cd4_bf5b_fdcc98bd30e6.slice/crio-4a788f0bddaa484b745c5a77e76a550e7c8cc239766d6a25a09a99469f2cb0b0 WatchSource:0}: Error finding container 4a788f0bddaa484b745c5a77e76a550e7c8cc239766d6a25a09a99469f2cb0b0: Status 404 returned error can't find the container with id 4a788f0bddaa484b745c5a77e76a550e7c8cc239766d6a25a09a99469f2cb0b0
Sep 15 21:04:21.304935 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:21.304897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x" event={"ID":"ea87473d-8756-4cd4-bf5b-fdcc98bd30e6","Type":"ContainerStarted","Data":"4a788f0bddaa484b745c5a77e76a550e7c8cc239766d6a25a09a99469f2cb0b0"}
Sep 15 21:04:25.082827 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.082797    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"]
Sep 15 21:04:25.085188 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.085171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.087831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.087796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xslrl\""
Sep 15 21:04:25.100026 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.100006    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"]
Sep 15 21:04:25.251922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.251883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.252085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.252015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkh8p\" (UniqueName: \"kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.332733 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.332702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x" event={"ID":"ea87473d-8756-4cd4-bf5b-fdcc98bd30e6","Type":"ContainerStarted","Data":"611b8209365e5163bed86f6d8b875f5bdb6e39d5da80f2c84b884f88b3a1e210"}
Sep 15 21:04:25.332955 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.332842    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:25.352929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.352903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkh8p\" (UniqueName: \"kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.353013 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.352944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.353339 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.353318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.356846 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.356805    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x" podStartSLOduration=2.185914544 podStartE2EDuration="5.356791571s" totalImagesPullingTime="3.170877027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:25.354969836 +0000 UTC m=+373.141337841" watchObservedRunningTime="2026-09-15 21:04:25.356791571 +0000 UTC m=+373.143159578"
Sep 15 21:04:25.366308 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.366282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkh8p\" (UniqueName: \"kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.396027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.396001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:25.703080 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:25.703048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"]
Sep 15 21:04:25.706286 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:25.706257    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8c3b1c_e9d3_41aa_9931_7b31bebd1523.slice/crio-391e80f8cb7164dde29e6a3dfd8573060457994391c0928921f460a0a6da4b13 WatchSource:0}: Error finding container 391e80f8cb7164dde29e6a3dfd8573060457994391c0928921f460a0a6da4b13: Status 404 returned error can't find the container with id 391e80f8cb7164dde29e6a3dfd8573060457994391c0928921f460a0a6da4b13
Sep 15 21:04:26.338629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:26.338593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp" event={"ID":"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523","Type":"ContainerStarted","Data":"391e80f8cb7164dde29e6a3dfd8573060457994391c0928921f460a0a6da4b13"}
Sep 15 21:04:26.907747 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:26.907681    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"]
Sep 15 21:04:26.910767 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:26.910748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:26.914223 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:26.914201    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4bftn\""
Sep 15 21:04:26.923772 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:26.923749    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"]
Sep 15 21:04:27.073123 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:27.073076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f92sf\" (UniqueName: \"kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf\") pod \"limitador-operator-controller-manager-74495c69f8-wbsgh\" (UID: \"596c1399-7b8b-4edd-b1d8-4bab9c5221b7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:27.174517 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:27.174427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f92sf\" (UniqueName: \"kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf\") pod \"limitador-operator-controller-manager-74495c69f8-wbsgh\" (UID: \"596c1399-7b8b-4edd-b1d8-4bab9c5221b7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:27.189423 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:27.189391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f92sf\" (UniqueName: \"kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf\") pod \"limitador-operator-controller-manager-74495c69f8-wbsgh\" (UID: \"596c1399-7b8b-4edd-b1d8-4bab9c5221b7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:27.223084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:27.223044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:27.522902 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:27.522862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"]
Sep 15 21:04:27.526705 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:27.526673    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596c1399_7b8b_4edd_b1d8_4bab9c5221b7.slice/crio-a4282f619f3c00ef548c35f104b4a710abf44a7eeeb0c443caef9830274bfb3d WatchSource:0}: Error finding container a4282f619f3c00ef548c35f104b4a710abf44a7eeeb0c443caef9830274bfb3d: Status 404 returned error can't find the container with id a4282f619f3c00ef548c35f104b4a710abf44a7eeeb0c443caef9830274bfb3d
Sep 15 21:04:28.353232 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:28.353197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh" event={"ID":"596c1399-7b8b-4edd-b1d8-4bab9c5221b7","Type":"ContainerStarted","Data":"a4282f619f3c00ef548c35f104b4a710abf44a7eeeb0c443caef9830274bfb3d"}
Sep 15 21:04:30.372443 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:30.372407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp" event={"ID":"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523","Type":"ContainerStarted","Data":"38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041"}
Sep 15 21:04:30.372880 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:30.372509    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:30.393927 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:30.393870    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp" podStartSLOduration=1.43986716 podStartE2EDuration="5.393848039s" totalImagesPullingTime="3.953980879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:30.391643611 +0000 UTC m=+378.178011618" watchObservedRunningTime="2026-09-15 21:04:30.393848039 +0000 UTC m=+378.180216045"
Sep 15 21:04:31.378811 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:31.378765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh" event={"ID":"596c1399-7b8b-4edd-b1d8-4bab9c5221b7","Type":"ContainerStarted","Data":"1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6"}
Sep 15 21:04:31.379215 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:31.378960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:31.400925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:31.400871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh" podStartSLOduration=2.4334280870000002 podStartE2EDuration="5.400856336s" totalImagesPullingTime="2.967428249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:31.399281037 +0000 UTC m=+379.185649046" watchObservedRunningTime="2026-09-15 21:04:31.400856336 +0000 UTC m=+379.187224414"
Sep 15 21:04:36.340780 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:36.340747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6qw9x"
Sep 15 21:04:39.370434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.370380    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-854d856c-9v9px" podUID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" containerName="console" containerID="cri-o://c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e" gracePeriod=15
Sep 15 21:04:39.617231 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.617210    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:04:39.683286 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683286 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683250    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjrbw\" (UniqueName: \"kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683289    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683399    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683428    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683631 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683450    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683631 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert\") pod \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\" (UID: \"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7\") "
Sep 15 21:04:39.683795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683764    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:04:39.683910 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config" (OuterVolumeSpecName: "console-config") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:04:39.683910 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683852    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-trusted-ca-bundle\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.683910 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:04:39.683910 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.683905    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:04:39.685622 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.685595    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw" (OuterVolumeSpecName: "kube-api-access-sjrbw") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "kube-api-access-sjrbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:04:39.685727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.685640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:04:39.685893 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.685876    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" (UID: "bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:04:39.784720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784685    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.784720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784710    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-oauth-config\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.784720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784719    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.784720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784729    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-oauth-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.785053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784738    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-console-serving-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:39.785053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:39.784748    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjrbw\" (UniqueName: \"kubernetes.io/projected/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7-kube-api-access-sjrbw\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:40.421370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.421336    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" containerID="c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e" exitCode=0
Sep 15 21:04:40.421830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.421382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-854d856c-9v9px" event={"ID":"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7","Type":"ContainerDied","Data":"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"}
Sep 15 21:04:40.421830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.421385    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-854d856c-9v9px"
Sep 15 21:04:40.421830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.421420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-854d856c-9v9px" event={"ID":"bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7","Type":"ContainerDied","Data":"d15c72e98d61b7b4baf9fd3c652d67f07fa047332e6d4e669fa48b56c1abede0"}
Sep 15 21:04:40.421830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.421439    2577 scope.go:117] "RemoveContainer" containerID="c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"
Sep 15 21:04:40.430894 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.430876    2577 scope.go:117] "RemoveContainer" containerID="c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"
Sep 15 21:04:40.431182 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:40.431161    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e\": container with ID starting with c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e not found: ID does not exist" containerID="c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"
Sep 15 21:04:40.431251 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.431197    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e"} err="failed to get container status \"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e\": rpc error: code = NotFound desc = could not find container \"c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e\": container with ID starting with c7e317f739008cd91092b490c09c12f2856ab183ef3a80115afe254c2894942e not found: ID does not exist"
Sep 15 21:04:40.444576 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.444540    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:04:40.448479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.448459    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-854d856c-9v9px"]
Sep 15 21:04:40.695985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:40.695910    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" path="/var/lib/kubelet/pods/bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7/volumes"
Sep 15 21:04:41.381750 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:41.381713    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:42.388118 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:42.388079    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.487794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.487762    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"]
Sep 15 21:04:43.488177 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.487930    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" containerName="manager" containerID="cri-o://38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041" gracePeriod=2
Sep 15 21:04:43.495862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.495802    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"]
Sep 15 21:04:43.497676 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.497643    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\": pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.509055 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.509027    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"]
Sep 15 21:04:43.509233 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.509212    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" containerName="manager" containerID="cri-o://1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6" gracePeriod=2
Sep 15 21:04:43.512775 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.512742    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.522984 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.522949    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:04:43.523509 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523485    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" containerName="console"
Sep 15 21:04:43.523509 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523506    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" containerName="console"
Sep 15 21:04:43.523661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523529    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" containerName="manager"
Sep 15 21:04:43.523661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523537    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" containerName="manager"
Sep 15 21:04:43.523729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523677    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8d35fa-ef7a-49b7-aa45-6bb947a2c5f7" containerName="console"
Sep 15 21:04:43.523729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.523690    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" containerName="manager"
Sep 15 21:04:43.527522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.527501    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"]
Sep 15 21:04:43.527639 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.527611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.538255 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.538231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:04:43.545668 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.545640    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"]
Sep 15 21:04:43.545972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.545960    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" containerName="manager"
Sep 15 21:04:43.546018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.545975    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" containerName="manager"
Sep 15 21:04:43.546061 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.546042    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" containerName="manager"
Sep 15 21:04:43.550318 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.550296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:43.565115 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.565081    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.567127 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.567010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"]
Sep 15 21:04:43.567127 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.567059    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.568896 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.568869    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.589850 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.589815    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.618470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.618438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbfts\" (UniqueName: \"kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.618596 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.618542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbdh\" (UniqueName: \"kubernetes.io/projected/6730d897-7607-4a6f-872a-65c3f8104fdf-kube-api-access-5tbdh\") pod \"limitador-operator-controller-manager-74495c69f8-fkpmb\" (UID: \"6730d897-7607-4a6f-872a-65c3f8104fdf\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:43.618655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.618629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.719319 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.719285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbdh\" (UniqueName: \"kubernetes.io/projected/6730d897-7607-4a6f-872a-65c3f8104fdf-kube-api-access-5tbdh\") pod \"limitador-operator-controller-manager-74495c69f8-fkpmb\" (UID: \"6730d897-7607-4a6f-872a-65c3f8104fdf\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:43.719486 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.719336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.719486 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.719441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbfts\" (UniqueName: \"kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.719815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.719790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.730822 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.730796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbdh\" (UniqueName: \"kubernetes.io/projected/6730d897-7607-4a6f-872a-65c3f8104fdf-kube-api-access-5tbdh\") pod \"limitador-operator-controller-manager-74495c69f8-fkpmb\" (UID: \"6730d897-7607-4a6f-872a-65c3f8104fdf\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:43.737932 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.737880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbfts\" (UniqueName: \"kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl5zk\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.755010 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.754992    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.757234 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.757218    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.757940 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.757918    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.760278 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.760260    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.762633 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.762610    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:43.764982 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:43.764961    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:43.820012 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.819984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f92sf\" (UniqueName: \"kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf\") pod \"596c1399-7b8b-4edd-b1d8-4bab9c5221b7\" (UID: \"596c1399-7b8b-4edd-b1d8-4bab9c5221b7\") "
Sep 15 21:04:43.820161 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.820068    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkh8p\" (UniqueName: \"kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p\") pod \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") "
Sep 15 21:04:43.820208 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.820163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume\") pod \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\" (UID: \"4b8c3b1c-e9d3-41aa-9931-7b31bebd1523\") "
Sep 15 21:04:43.820698 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.820672    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" (UID: "4b8c3b1c-e9d3-41aa-9931-7b31bebd1523"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:04:43.822345 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.822323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf" (OuterVolumeSpecName: "kube-api-access-f92sf") pod "596c1399-7b8b-4edd-b1d8-4bab9c5221b7" (UID: "596c1399-7b8b-4edd-b1d8-4bab9c5221b7"). InnerVolumeSpecName "kube-api-access-f92sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:04:43.822399 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.822345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p" (OuterVolumeSpecName: "kube-api-access-jkh8p") pod "4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" (UID: "4b8c3b1c-e9d3-41aa-9931-7b31bebd1523"). InnerVolumeSpecName "kube-api-access-jkh8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:04:43.918860 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.918828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:43.921216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.921196    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-extensions-socket-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:43.921288 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.921219    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f92sf\" (UniqueName: \"kubernetes.io/projected/596c1399-7b8b-4edd-b1d8-4bab9c5221b7-kube-api-access-f92sf\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:43.921288 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.921229    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkh8p\" (UniqueName: \"kubernetes.io/projected/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523-kube-api-access-jkh8p\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:04:43.926169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:43.926150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:44.201248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.201176    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:04:44.202772 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:44.202743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c31d3e8_393d_44a5_adc2_a0b1cef427d0.slice/crio-7929b8165677c7a8d82aab56f107dfc3e2fd48e28434c378f12ed16fbe986a92 WatchSource:0}: Error finding container 7929b8165677c7a8d82aab56f107dfc3e2fd48e28434c378f12ed16fbe986a92: Status 404 returned error can't find the container with id 7929b8165677c7a8d82aab56f107dfc3e2fd48e28434c378f12ed16fbe986a92
Sep 15 21:04:44.202889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.202826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"]
Sep 15 21:04:44.205325 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:04:44.205304    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6730d897_7607_4a6f_872a_65c3f8104fdf.slice/crio-eaa39e7dbe79e14446026ecd38f97bb69f175663a4766fc22e090db69f980b26 WatchSource:0}: Error finding container eaa39e7dbe79e14446026ecd38f97bb69f175663a4766fc22e090db69f980b26: Status 404 returned error can't find the container with id eaa39e7dbe79e14446026ecd38f97bb69f175663a4766fc22e090db69f980b26
Sep 15 21:04:44.445213 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.445179    2577 generic.go:356] "Generic (PLEG): container finished" podID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" containerID="1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6" exitCode=0
Sep 15 21:04:44.445390 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.445238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:44.445390 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.445287    2577 scope.go:117] "RemoveContainer" containerID="1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6"
Sep 15 21:04:44.447200 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.447170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb" event={"ID":"6730d897-7607-4a6f-872a-65c3f8104fdf","Type":"ContainerStarted","Data":"1b3b5d1a549830f4acfeb71e0457cb9738d3ac962d99ca747345b4fc5063999c"}
Sep 15 21:04:44.447327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.447238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb" event={"ID":"6730d897-7607-4a6f-872a-65c3f8104fdf","Type":"ContainerStarted","Data":"eaa39e7dbe79e14446026ecd38f97bb69f175663a4766fc22e090db69f980b26"}
Sep 15 21:04:44.447388 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.447345    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:44.447686 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.447650    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:44.451799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.449520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" event={"ID":"1c31d3e8-393d-44a5-adc2-a0b1cef427d0","Type":"ContainerStarted","Data":"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"}
Sep 15 21:04:44.451799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.449653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" event={"ID":"1c31d3e8-393d-44a5-adc2-a0b1cef427d0","Type":"ContainerStarted","Data":"7929b8165677c7a8d82aab56f107dfc3e2fd48e28434c378f12ed16fbe986a92"}
Sep 15 21:04:44.451799 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.450670    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:44.451799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.450861    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:04:44.453014 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.452986    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:44.457118 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.457091    2577 generic.go:356] "Generic (PLEG): container finished" podID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" containerID="38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041" exitCode=0
Sep 15 21:04:44.457342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.457310    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:44.462827 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.462805    2577 scope.go:117] "RemoveContainer" containerID="1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6"
Sep 15 21:04:44.463091 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.463074    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6\": container with ID starting with 1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6 not found: ID does not exist" containerID="1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6"
Sep 15 21:04:44.463162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.463097    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6"} err="failed to get container status \"1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6\": rpc error: code = NotFound desc = could not find container \"1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6\": container with ID starting with 1e231d426ba0b4bf541f543d43f1be6c4d2370aed99a6e78ca07baa08545eac6 not found: ID does not exist"
Sep 15 21:04:44.463162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.463113    2577 scope.go:117] "RemoveContainer" containerID="38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041"
Sep 15 21:04:44.474679 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.474654    2577 scope.go:117] "RemoveContainer" containerID="38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041"
Sep 15 21:04:44.474811 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.474786    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:44.475005 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.474984    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041\": container with ID starting with 38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041 not found: ID does not exist" containerID="38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041"
Sep 15 21:04:44.475069 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.475013    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041"} err="failed to get container status \"38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041\": rpc error: code = NotFound desc = could not find container \"38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041\": container with ID starting with 38a9f72e9b88a3cd081bd158d4a1dbb67b56d8fac35317bc489a8f0d1228b041 not found: ID does not exist"
Sep 15 21:04:44.475545 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.475507    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb" podStartSLOduration=1.475494424 podStartE2EDuration="1.475494424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:44.472806977 +0000 UTC m=+392.259175000" watchObservedRunningTime="2026-09-15 21:04:44.475494424 +0000 UTC m=+392.261862431"
Sep 15 21:04:44.495448 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.495404    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" podStartSLOduration=1.495390027 podStartE2EDuration="1.495390027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:44.494688615 +0000 UTC m=+392.281056626" watchObservedRunningTime="2026-09-15 21:04:44.495390027 +0000 UTC m=+392.281758033"
Sep 15 21:04:44.496941 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.496916    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-qlrfp\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qlrfp"
Sep 15 21:04:44.498917 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:04:44.498889    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-wbsgh\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wbsgh"
Sep 15 21:04:44.691087 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.691055    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b8c3b1c-e9d3-41aa-9931-7b31bebd1523" path="/var/lib/kubelet/pods/4b8c3b1c-e9d3-41aa-9931-7b31bebd1523/volumes"
Sep 15 21:04:44.691431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:44.691417    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596c1399-7b8b-4edd-b1d8-4bab9c5221b7" path="/var/lib/kubelet/pods/596c1399-7b8b-4edd-b1d8-4bab9c5221b7/volumes"
Sep 15 21:04:55.466662 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:55.466629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fkpmb"
Sep 15 21:04:56.471753 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:04:56.471721    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:05:00.102286 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.102247    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:05:00.102753 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.102440    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" podUID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" containerName="manager" containerID="cri-o://77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529" gracePeriod=10
Sep 15 21:05:00.341885 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.341861    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:05:00.458177 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.458101    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:05:00.458504 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.458491    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" containerName="manager"
Sep 15 21:05:00.458546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.458507    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" containerName="manager"
Sep 15 21:05:00.458611 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.458598    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" containerName="manager"
Sep 15 21:05:00.460622 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.460597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.475316 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.475288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbfts\" (UniqueName: \"kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts\") pod \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") "
Sep 15 21:05:00.475411 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.475334    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume\") pod \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\" (UID: \"1c31d3e8-393d-44a5-adc2-a0b1cef427d0\") "
Sep 15 21:05:00.475743 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.475721    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1c31d3e8-393d-44a5-adc2-a0b1cef427d0" (UID: "1c31d3e8-393d-44a5-adc2-a0b1cef427d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:05:00.477409 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.477387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts" (OuterVolumeSpecName: "kube-api-access-rbfts") pod "1c31d3e8-393d-44a5-adc2-a0b1cef427d0" (UID: "1c31d3e8-393d-44a5-adc2-a0b1cef427d0"). InnerVolumeSpecName "kube-api-access-rbfts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:05:00.482181 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.482160    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:05:00.541676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.541641    2577 generic.go:356] "Generic (PLEG): container finished" podID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" containerID="77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529" exitCode=0
Sep 15 21:05:00.541676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.541686    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"
Sep 15 21:05:00.541872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.541725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" event={"ID":"1c31d3e8-393d-44a5-adc2-a0b1cef427d0","Type":"ContainerDied","Data":"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"}
Sep 15 21:05:00.541872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.541766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk" event={"ID":"1c31d3e8-393d-44a5-adc2-a0b1cef427d0","Type":"ContainerDied","Data":"7929b8165677c7a8d82aab56f107dfc3e2fd48e28434c378f12ed16fbe986a92"}
Sep 15 21:05:00.541872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.541793    2577 scope.go:117] "RemoveContainer" containerID="77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"
Sep 15 21:05:00.550500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.550484    2577 scope.go:117] "RemoveContainer" containerID="77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"
Sep 15 21:05:00.550784 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:05:00.550766    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529\": container with ID starting with 77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529 not found: ID does not exist" containerID="77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"
Sep 15 21:05:00.550852 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.550818    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529"} err="failed to get container status \"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529\": rpc error: code = NotFound desc = could not find container \"77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529\": container with ID starting with 77ae189aa0f85cb49123fd1a105aa01bbc15df83193b93853e7b76465aedb529 not found: ID does not exist"
Sep 15 21:05:00.572877 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.572850    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:05:00.576800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.576773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.576905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.576809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g9tt\" (UniqueName: \"kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.576905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.576872    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbfts\" (UniqueName: \"kubernetes.io/projected/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-kube-api-access-rbfts\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:05:00.576905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.576883    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c31d3e8-393d-44a5-adc2-a0b1cef427d0-extensions-socket-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:05:00.577118 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.577082    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl5zk"]
Sep 15 21:05:00.677456 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.677424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.677687 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.677464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g9tt\" (UniqueName: \"kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.677878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.677853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.688331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.688304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g9tt\" (UniqueName: \"kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f7rc2\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:00.693905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.693881    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c31d3e8-393d-44a5-adc2-a0b1cef427d0" path="/var/lib/kubelet/pods/1c31d3e8-393d-44a5-adc2-a0b1cef427d0/volumes"
Sep 15 21:05:00.771161 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:00.771103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:01.060596 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:01.060558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:05:01.063016 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:05:01.062989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b9f543_66fe_4f52_bb78_0319f0c2c48f.slice/crio-f36c16a2546ebd57f376151b12102999e0619f5a159dcf16f5b8ebb0d992ae67 WatchSource:0}: Error finding container f36c16a2546ebd57f376151b12102999e0619f5a159dcf16f5b8ebb0d992ae67: Status 404 returned error can't find the container with id f36c16a2546ebd57f376151b12102999e0619f5a159dcf16f5b8ebb0d992ae67
Sep 15 21:05:01.549960 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:01.549928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" event={"ID":"04b9f543-66fe-4f52-bb78-0319f0c2c48f","Type":"ContainerStarted","Data":"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"}
Sep 15 21:05:01.550348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:01.549978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" event={"ID":"04b9f543-66fe-4f52-bb78-0319f0c2c48f","Type":"ContainerStarted","Data":"f36c16a2546ebd57f376151b12102999e0619f5a159dcf16f5b8ebb0d992ae67"}
Sep 15 21:05:01.550348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:01.550060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:01.600772 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:01.600701    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" podStartSLOduration=1.600677527 podStartE2EDuration="1.600677527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:05:01.597320752 +0000 UTC m=+409.383688762" watchObservedRunningTime="2026-09-15 21:05:01.600677527 +0000 UTC m=+409.387045536"
Sep 15 21:05:12.557756 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:12.557720    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:05:16.517055 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.517023    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"]
Sep 15 21:05:16.522800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.522780    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.525430 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.525410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vtglk\""
Sep 15 21:05:16.531705 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.531681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"]
Sep 15 21:05:16.620347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0678c506-cf08-4e7b-a97d-ccda27c4803a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch96b\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-kube-api-access-ch96b\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.620888 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.620849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721374 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0678c506-cf08-4e7b-a97d-ccda27c4803a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch96b\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-kube-api-access-ch96b\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.721867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.722019 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.721902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.722076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.722017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.722076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.722053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.722168 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.722151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.722627 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.722604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0678c506-cf08-4e7b-a97d-ccda27c4803a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.723867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.723852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.724123 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.724098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.729273 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.729250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.729603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.729583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch96b\" (UniqueName: \"kubernetes.io/projected/0678c506-cf08-4e7b-a97d-ccda27c4803a-kube-api-access-ch96b\") pod \"maas-default-gateway-openshift-default-5ccb968c66-q5rp9\" (UID: \"0678c506-cf08-4e7b-a97d-ccda27c4803a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:16.834585 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:16.834486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:17.112974 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.112952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"]
Sep 15 21:05:17.115493 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:05:17.115462    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0678c506_cf08_4e7b_a97d_ccda27c4803a.slice/crio-851a83cad75387bc26bfb306a59bf90d67f76f193d11819cb36b6ce37b5b7329 WatchSource:0}: Error finding container 851a83cad75387bc26bfb306a59bf90d67f76f193d11819cb36b6ce37b5b7329: Status 404 returned error can't find the container with id 851a83cad75387bc26bfb306a59bf90d67f76f193d11819cb36b6ce37b5b7329
Sep 15 21:05:17.117771 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.117737    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:05:17.117841 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.117804    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:05:17.117886 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.117837    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:05:17.631699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.631660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" event={"ID":"0678c506-cf08-4e7b-a97d-ccda27c4803a","Type":"ContainerStarted","Data":"28dcf93988f2a5e4d0202a25828463e919f11a6970491798570a1f3ca82d7a16"}
Sep 15 21:05:17.632087 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.631725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" event={"ID":"0678c506-cf08-4e7b-a97d-ccda27c4803a","Type":"ContainerStarted","Data":"851a83cad75387bc26bfb306a59bf90d67f76f193d11819cb36b6ce37b5b7329"}
Sep 15 21:05:17.652048 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.651996    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podStartSLOduration=1.651975669 podStartE2EDuration="1.651975669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:05:17.651428245 +0000 UTC m=+425.437796255" watchObservedRunningTime="2026-09-15 21:05:17.651975669 +0000 UTC m=+425.438343679"
Sep 15 21:05:17.834825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:17.834791    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:18.839876 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:18.839845    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:19.641604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:19.641557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:05:19.642680 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:05:19.642663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:06:48.058636 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.058605    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"]
Sep 15 21:06:48.062476 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.062456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.065057 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.065039    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 21:06:48.065154 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.065138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n8k8b\""
Sep 15 21:06:48.071929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.071906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"]
Sep 15 21:06:48.132348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.132311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh7rg\" (UniqueName: \"kubernetes.io/projected/31959955-2c6a-40d1-9735-0d5a17b933a0-kube-api-access-mh7rg\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.132348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.132350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31959955-2c6a-40d1-9735-0d5a17b933a0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.233420 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.233388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh7rg\" (UniqueName: \"kubernetes.io/projected/31959955-2c6a-40d1-9735-0d5a17b933a0-kube-api-access-mh7rg\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.233625 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.233430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31959955-2c6a-40d1-9735-0d5a17b933a0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.234196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.234170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31959955-2c6a-40d1-9735-0d5a17b933a0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.244920 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.244891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh7rg\" (UniqueName: \"kubernetes.io/projected/31959955-2c6a-40d1-9735-0d5a17b933a0-kube-api-access-mh7rg\") pod \"payload-processing-5b54b7fbb7-8mlr4\" (UID: \"31959955-2c6a-40d1-9735-0d5a17b933a0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.375103 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.375016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:48.646611 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:48.646551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"]
Sep 15 21:06:48.649107 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:06:48.648991    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31959955_2c6a_40d1_9735_0d5a17b933a0.slice/crio-6c08e9c21d7aaf095e168d3e5f3bdfd553e69de6e43ceec8e691d08fd51c842a WatchSource:0}: Error finding container 6c08e9c21d7aaf095e168d3e5f3bdfd553e69de6e43ceec8e691d08fd51c842a: Status 404 returned error can't find the container with id 6c08e9c21d7aaf095e168d3e5f3bdfd553e69de6e43ceec8e691d08fd51c842a
Sep 15 21:06:49.064716 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:49.064682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4" event={"ID":"31959955-2c6a-40d1-9735-0d5a17b933a0","Type":"ContainerStarted","Data":"6c08e9c21d7aaf095e168d3e5f3bdfd553e69de6e43ceec8e691d08fd51c842a"}
Sep 15 21:06:51.074781 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:51.074749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4" event={"ID":"31959955-2c6a-40d1-9735-0d5a17b933a0","Type":"ContainerStarted","Data":"d8c0d579a36ac0e91d756b1e5799ba3e84057a7e4f1e6306d09e6b312494104b"}
Sep 15 21:06:51.075170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:51.074818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:06:51.091721 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:51.091673    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4" podStartSLOduration=1.6822042910000001 podStartE2EDuration="3.091656093s" totalImagesPullingTime="1.409451802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:06:51.090749351 +0000 UTC m=+518.877117372" watchObservedRunningTime="2026-09-15 21:06:51.091656093 +0000 UTC m=+518.878024102"
Sep 15 21:06:58.114741 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.114708    2577 generic.go:356] "Generic (PLEG): container finished" podID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerID="28dcf93988f2a5e4d0202a25828463e919f11a6970491798570a1f3ca82d7a16" exitCode=0
Sep 15 21:06:58.115188 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.114754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" event={"ID":"0678c506-cf08-4e7b-a97d-ccda27c4803a","Type":"ContainerDied","Data":"28dcf93988f2a5e4d0202a25828463e919f11a6970491798570a1f3ca82d7a16"}
Sep 15 21:06:58.115188 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.115093    2577 scope.go:117] "RemoveContainer" containerID="28dcf93988f2a5e4d0202a25828463e919f11a6970491798570a1f3ca82d7a16"
Sep 15 21:06:58.116098 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.116070    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:06:58.116169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.116133    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:06:58.116169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:58.116162    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:06:59.121522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:59.121484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" event={"ID":"0678c506-cf08-4e7b-a97d-ccda27c4803a","Type":"ContainerStarted","Data":"2c18d5e37580c0990320f66bffcb04cad5ece3b10ea651306437b191f99c2bd1"}
Sep 15 21:06:59.835246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:59.835216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:06:59.836508 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:59.836463    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-q5rp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 21:06:59.836664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:06:59.836550    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podUID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 21:07:00.835861 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:00.835818    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-q5rp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 21:07:00.836342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:00.835883    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podUID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 21:07:01.835254 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:01.835220    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-q5rp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 21:07:01.835581 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:01.835521    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podUID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 21:07:02.081937 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:02.081904    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8mlr4"
Sep 15 21:07:02.835901 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:02.835860    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-q5rp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 21:07:02.836107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:02.835929    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podUID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 21:07:03.835321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:03.835288    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-q5rp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 15 21:07:03.836267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:03.835874    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9" podUID="0678c506-cf08-4e7b-a97d-ccda27c4803a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 15 21:07:04.642494 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:04.642438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:07:04.839663 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:04.839629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:07:05.161878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:05.161853    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-q5rp9"
Sep 15 21:07:47.883720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.883688    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"]
Sep 15 21:07:47.887417 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.887396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.891684 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.891660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 21:07:47.892793 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.892768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 15 21:07:47.892919 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.892799    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 21:07:47.893161 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.893141    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b2fk7\""
Sep 15 21:07:47.901840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.900845    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"]
Sep 15 21:07:47.991982 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.991944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.992170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.991992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhgp\" (UniqueName: \"kubernetes.io/projected/3a7631f7-401c-44d9-8ae3-374efb6b85f5-kube-api-access-cqhgp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.992170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.992048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.992170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.992084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.992170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.992107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:47.992346 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:47.992236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093124 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093297 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093297 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhgp\" (UniqueName: \"kubernetes.io/projected/3a7631f7-401c-44d9-8ae3-374efb6b85f5-kube-api-access-cqhgp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093453 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093501 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.093871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.093856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.095759 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.095730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a7631f7-401c-44d9-8ae3-374efb6b85f5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.095999 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.095984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a7631f7-401c-44d9-8ae3-374efb6b85f5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.103464 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.103431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhgp\" (UniqueName: \"kubernetes.io/projected/3a7631f7-401c-44d9-8ae3-374efb6b85f5-kube-api-access-cqhgp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8\" (UID: \"3a7631f7-401c-44d9-8ae3-374efb6b85f5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.201383 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.201306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:48.478015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:48.477951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"]
Sep 15 21:07:48.480642 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:07:48.480613    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7631f7_401c_44d9_8ae3_374efb6b85f5.slice/crio-f68ffc35d36da14ba0528e27f5037511b3038b3845f05bc24c9276c42d95c849 WatchSource:0}: Error finding container f68ffc35d36da14ba0528e27f5037511b3038b3845f05bc24c9276c42d95c849: Status 404 returned error can't find the container with id f68ffc35d36da14ba0528e27f5037511b3038b3845f05bc24c9276c42d95c849
Sep 15 21:07:49.369070 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:49.369033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8" event={"ID":"3a7631f7-401c-44d9-8ae3-374efb6b85f5","Type":"ContainerStarted","Data":"f68ffc35d36da14ba0528e27f5037511b3038b3845f05bc24c9276c42d95c849"}
Sep 15 21:07:50.379339 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:50.379305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8" event={"ID":"3a7631f7-401c-44d9-8ae3-374efb6b85f5","Type":"ContainerStarted","Data":"ed3a187462b09a7e9de3c2ed38f7e03ccbe446b37c70d725a36af09395fe8de1"}
Sep 15 21:07:50.398750 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:50.398701    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8" podStartSLOduration=2.383295532 podStartE2EDuration="3.398684604s" totalImagesPullingTime="1.015389072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:07:50.397992658 +0000 UTC m=+578.184360668" watchObservedRunningTime="2026-09-15 21:07:50.398684604 +0000 UTC m=+578.185052611"
Sep 15 21:07:58.201399 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:58.201366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:58.201871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:58.201421    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:58.214163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:58.214134    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:07:58.431195 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:07:58.431166    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8"
Sep 15 21:08:12.619419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:12.619385    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:08:12.620109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:12.620087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:08:20.499018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.498983    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"]
Sep 15 21:08:20.503191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.503171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.506063 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.506042    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 15 21:08:20.513928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.513902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"]
Sep 15 21:08:20.600458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.600640 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.600640 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.600640 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9j8x\" (UniqueName: \"kubernetes.io/projected/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-kube-api-access-z9j8x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.600749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.600749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.600683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.701907 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.701878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.701907 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.701909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.701955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9j8x\" (UniqueName: \"kubernetes.io/projected/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-kube-api-access-z9j8x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.701982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.701999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.702020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702362 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.702319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702423 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.702374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.702511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.702490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.704373 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.704350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.704648 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.704630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.712985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.712960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9j8x\" (UniqueName: \"kubernetes.io/projected/63b0bee9-dd9d-4e8f-951f-4e818ac04a3f-kube-api-access-z9j8x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml\" (UID: \"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:20.814736 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:20.814646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:21.093136 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:21.093114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"]
Sep 15 21:08:21.095527 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:08:21.095501    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b0bee9_dd9d_4e8f_951f_4e818ac04a3f.slice/crio-5d6dcbd1938369a80229be6d03133b140777c2f95a81e31d9eeecaecb9dbd577 WatchSource:0}: Error finding container 5d6dcbd1938369a80229be6d03133b140777c2f95a81e31d9eeecaecb9dbd577: Status 404 returned error can't find the container with id 5d6dcbd1938369a80229be6d03133b140777c2f95a81e31d9eeecaecb9dbd577
Sep 15 21:08:21.096969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:21.096952    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:08:21.545442 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:21.545396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml" event={"ID":"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f","Type":"ContainerStarted","Data":"3a132248995110c5aad49953e37f4aa79106d7a078e2bd9e12a3971927ac3ad9"}
Sep 15 21:08:21.545913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:21.545477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml" event={"ID":"63b0bee9-dd9d-4e8f-951f-4e818ac04a3f","Type":"ContainerStarted","Data":"5d6dcbd1938369a80229be6d03133b140777c2f95a81e31d9eeecaecb9dbd577"}
Sep 15 21:08:21.566541 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:21.566488    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml" podStartSLOduration=1.386733456 podStartE2EDuration="1.566473025s" totalImagesPullingTime="179.739569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:08:21.564826999 +0000 UTC m=+609.351195008" watchObservedRunningTime="2026-09-15 21:08:21.566473025 +0000 UTC m=+609.352841067"
Sep 15 21:08:30.815107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:30.815073    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:30.815491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:30.815363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:30.828026 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:30.828002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:08:31.604449 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:08:31.604415    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml"
Sep 15 21:10:17.264812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.264777    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:10:17.268438 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.268421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.271117 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.271093    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-dca193-openshift-default-dockercfg-sxs42\""
Sep 15 21:10:17.287830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.287806    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:10:17.357634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.357606    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:10:17.361084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.361067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.364012 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.363983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-0df719-openshift-default-dockercfg-t8r8t\""
Sep 15 21:10:17.369266 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369366 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369428 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369590 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369590 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9rqt\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.369769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.369665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.374134 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.374111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:10:17.471158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m8k9\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.471342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.471342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.471645 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.471645 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9rqt\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471837 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471919 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.471982 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.471931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.472060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.472162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472397 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472463 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.472529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.472529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.472678 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.472840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.472749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.473814 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.473780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.474712 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.474692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.483676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.483648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.484246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.484227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9rqt\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt\") pod \"e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") " pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.573405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m8k9\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.573985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.573938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.574083 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.574002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.574083 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.574049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.574456 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.574432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.576201 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.576177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.576340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.576294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.578309 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.578289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:17.588621 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.588596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.589363 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.589318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m8k9\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9\") pod \"e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") " pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.673746 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.673708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:17.870851 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.870826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:10:17.874016 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:17.873971    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c7a38b_af66_4a5e_9180_2451858939da.slice/crio-3178b80181fa4e6ff7102e2eeab393637c40e0ebadd5394412d0e52ab36ce668 WatchSource:0}: Error finding container 3178b80181fa4e6ff7102e2eeab393637c40e0ebadd5394412d0e52ab36ce668: Status 404 returned error can't find the container with id 3178b80181fa4e6ff7102e2eeab393637c40e0ebadd5394412d0e52ab36ce668
Sep 15 21:10:17.876246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.876216    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:17.876314 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.876288    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:17.876352 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.876320    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:17.955740 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.955717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:10:17.957634 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:17.957605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ab8fba_290f_4b94_b3d2_ab3387f73c6a.slice/crio-4861f0b678c1383189649e6fdfb2c6e00782c23d8673259bf7a7fbbea6e66a04 WatchSource:0}: Error finding container 4861f0b678c1383189649e6fdfb2c6e00782c23d8673259bf7a7fbbea6e66a04: Status 404 returned error can't find the container with id 4861f0b678c1383189649e6fdfb2c6e00782c23d8673259bf7a7fbbea6e66a04
Sep 15 21:10:17.960342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.960307    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:17.960449 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.960376    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:17.960449 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:17.960408    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:18.132385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.132295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerStarted","Data":"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"}
Sep 15 21:10:18.132385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.132362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerStarted","Data":"4861f0b678c1383189649e6fdfb2c6e00782c23d8673259bf7a7fbbea6e66a04"}
Sep 15 21:10:18.134685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.134647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerStarted","Data":"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"}
Sep 15 21:10:18.134825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.134699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerStarted","Data":"3178b80181fa4e6ff7102e2eeab393637c40e0ebadd5394412d0e52ab36ce668"}
Sep 15 21:10:18.158589 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.158504    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podStartSLOduration=1.158485054 podStartE2EDuration="1.158485054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:18.157661124 +0000 UTC m=+725.944029133" watchObservedRunningTime="2026-09-15 21:10:18.158485054 +0000 UTC m=+725.944853064"
Sep 15 21:10:18.180256 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.180193    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podStartSLOduration=1.180173319 podStartE2EDuration="1.180173319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:18.180151782 +0000 UTC m=+725.966519791" watchObservedRunningTime="2026-09-15 21:10:18.180173319 +0000 UTC m=+725.966541328"
Sep 15 21:10:18.578500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.578466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:18.583461 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.583436    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:18.674611 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:18.674548    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:19.139766 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:19.139734    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:19.140716 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:19.140697    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:19.679799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:19.679772    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:20.145783 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:20.145747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:20.146785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:20.146762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:22.365928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.365897    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:10:22.369557 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.369536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.372155 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.372133    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-dca193-dockercfg-l4jdm\""
Sep 15 21:10:22.372387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.372375    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-dca193\""
Sep 15 21:10:22.381351 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.381328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:10:22.527120 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.527081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.527120 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.527123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wjq\" (UniqueName: \"kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.628538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.628445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.628538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.628487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wjq\" (UniqueName: \"kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.629389 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.629356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.637252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.637226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wjq\" (UniqueName: \"kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq\") pod \"payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.683358 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.683324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:22.970131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:22.970004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:10:22.975417 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:22.975387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50ad04e_e377_486b_b310_e73fde8df4bb.slice/crio-6f5c2b94a5fd2bdef2e90555d07b653e746e3d01d1f2ffdaddc7f57bc5e96a31 WatchSource:0}: Error finding container 6f5c2b94a5fd2bdef2e90555d07b653e746e3d01d1f2ffdaddc7f57bc5e96a31: Status 404 returned error can't find the container with id 6f5c2b94a5fd2bdef2e90555d07b653e746e3d01d1f2ffdaddc7f57bc5e96a31
Sep 15 21:10:23.162154 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.162115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" event={"ID":"f50ad04e-e377-486b-b310-e73fde8df4bb","Type":"ContainerStarted","Data":"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"}
Sep 15 21:10:23.162333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.162172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" event={"ID":"f50ad04e-e377-486b-b310-e73fde8df4bb","Type":"ContainerStarted","Data":"6f5c2b94a5fd2bdef2e90555d07b653e746e3d01d1f2ffdaddc7f57bc5e96a31"}
Sep 15 21:10:23.162333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.162245    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:23.180708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.180661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" podStartSLOduration=1.180648505 podStartE2EDuration="1.180648505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:23.178466835 +0000 UTC m=+730.964834867" watchObservedRunningTime="2026-09-15 21:10:23.180648505 +0000 UTC m=+730.967016513"
Sep 15 21:10:23.823689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.823650    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:10:23.828274 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.828248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:23.832085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.832058    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0df719\""
Sep 15 21:10:23.839647 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.838931    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:10:23.939745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.939711    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:10:23.943324 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.943305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:23.947659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.947638    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0df719-dockercfg-wbtpr\""
Sep 15 21:10:23.961415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.961391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:10:23.975830 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.975803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmcq\" (UniqueName: \"kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:23.975931 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:23.975833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.077050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.076961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmcq\" (UniqueName: \"kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.077050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.077004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.077280 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.077071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr9wm\" (UniqueName: \"kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.077280 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.077158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.077838 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.077819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.087748 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.087720    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmcq\" (UniqueName: \"kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq\") pod \"payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.177973 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.177936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr9wm\" (UniqueName: \"kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.178164 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.178093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.178922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.178892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.180388 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.180362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:24.188810 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.188790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr9wm\" (UniqueName: \"kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm\") pod \"payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.254509 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.254434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:24.463809 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.463785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:10:24.466406 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:24.466374    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805620bd_5e2a_415b_9105_0b576f7ce3f1.slice/crio-16cdf6bfadd446c409c623a0fafe2707d529f91872b94d1a2468fe95853ccf9f WatchSource:0}: Error finding container 16cdf6bfadd446c409c623a0fafe2707d529f91872b94d1a2468fe95853ccf9f: Status 404 returned error can't find the container with id 16cdf6bfadd446c409c623a0fafe2707d529f91872b94d1a2468fe95853ccf9f
Sep 15 21:10:24.536717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:24.536671    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:10:24.539603 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:24.539558    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae73e26_25cc_4ea2_b81d_b793de1ed273.slice/crio-eea15632818cf6f9f9e040202c2d6f4cb83ba0dc3fddcdcf1bdea9afa3929b17 WatchSource:0}: Error finding container eea15632818cf6f9f9e040202c2d6f4cb83ba0dc3fddcdcf1bdea9afa3929b17: Status 404 returned error can't find the container with id eea15632818cf6f9f9e040202c2d6f4cb83ba0dc3fddcdcf1bdea9afa3929b17
Sep 15 21:10:25.175529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.175300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" event={"ID":"805620bd-5e2a-415b-9105-0b576f7ce3f1","Type":"ContainerStarted","Data":"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"}
Sep 15 21:10:25.175529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.175370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:25.175529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.175395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" event={"ID":"805620bd-5e2a-415b-9105-0b576f7ce3f1","Type":"ContainerStarted","Data":"16cdf6bfadd446c409c623a0fafe2707d529f91872b94d1a2468fe95853ccf9f"}
Sep 15 21:10:25.177165 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.177140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" event={"ID":"aae73e26-25cc-4ea2-b81d-b793de1ed273","Type":"ContainerStarted","Data":"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"}
Sep 15 21:10:25.177289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.177186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" event={"ID":"aae73e26-25cc-4ea2-b81d-b793de1ed273","Type":"ContainerStarted","Data":"eea15632818cf6f9f9e040202c2d6f4cb83ba0dc3fddcdcf1bdea9afa3929b17"}
Sep 15 21:10:25.177289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.177238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:25.269015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.268957    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" podStartSLOduration=2.268936704 podStartE2EDuration="2.268936704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:25.26864633 +0000 UTC m=+733.055014339" watchObservedRunningTime="2026-09-15 21:10:25.268936704 +0000 UTC m=+733.055304713"
Sep 15 21:10:25.271613 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:25.271555    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" podStartSLOduration=2.271541987 podStartE2EDuration="2.271541987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:25.219453379 +0000 UTC m=+733.005821387" watchObservedRunningTime="2026-09-15 21:10:25.271541987 +0000 UTC m=+733.057909995"
Sep 15 21:10:26.682456 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.682421    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:10:26.686616 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.686584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.690173 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.690144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bfe4d22a-openshift-default-dockercfg-hn67g\""
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.700955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phvrh\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.701537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.701409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.703945 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.703918    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:10:26.803056 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvrh\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.803594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.804369 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.803907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.804369 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.804249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.804592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.804503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.804592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.804544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.805213 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.805161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.806969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.806945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.808186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.808167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.811924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.811905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvrh\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:26.811924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:26.811917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token\") pod \"e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") " pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:27.001465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:27.001433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:27.286498 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:27.286475    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:10:27.288216 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:27.288188    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aa8f99_cd0b_44c4_8a70_56da82eb2f96.slice/crio-dccfba1c8950d5ec74c8c0b727d7a1ebfe044edc5ce23402558ae8eb7fb3d31e WatchSource:0}: Error finding container dccfba1c8950d5ec74c8c0b727d7a1ebfe044edc5ce23402558ae8eb7fb3d31e: Status 404 returned error can't find the container with id dccfba1c8950d5ec74c8c0b727d7a1ebfe044edc5ce23402558ae8eb7fb3d31e
Sep 15 21:10:27.290558 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:27.290526    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:27.290674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:27.290605    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:27.290674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:27.290635    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:28.194906 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:28.194869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerStarted","Data":"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"}
Sep 15 21:10:28.195379 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:28.194928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerStarted","Data":"dccfba1c8950d5ec74c8c0b727d7a1ebfe044edc5ce23402558ae8eb7fb3d31e"}
Sep 15 21:10:28.223722 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:28.223676    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podStartSLOduration=2.223659987 podStartE2EDuration="2.223659987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:28.222276749 +0000 UTC m=+736.008644758" watchObservedRunningTime="2026-09-15 21:10:28.223659987 +0000 UTC m=+736.010027994"
Sep 15 21:10:29.002582 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:29.002515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:29.010543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:29.010506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:29.202542 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:29.202509    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:29.203924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:29.203906    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:34.050594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.050242    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:10:34.067278 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.066552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.075727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.074875    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bfe4d22a\""
Sep 15 21:10:34.075727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.075143    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bfe4d22a-dockercfg-rftw6\""
Sep 15 21:10:34.076930 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.076298    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:10:34.080902 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.079471    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:10:34.097950 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.096232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.097950 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.096371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4hs6\" (UniqueName: \"kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.102996 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.102869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:10:34.103129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.102970    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.176546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.176508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:10:34.197148 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.197109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-876bm\" (UniqueName: \"kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.201713 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.201683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.202902 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.202877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4hs6\" (UniqueName: \"kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.203094 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.203077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.203312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.202742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.212591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.212544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4hs6\" (UniqueName: \"kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6\") pod \"payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.304288 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.304182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.304745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.304715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-876bm\" (UniqueName: \"kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.305119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.305058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.314214 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.314192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-876bm\" (UniqueName: \"kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm\") pod \"payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") " pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.387100 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.387069    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:34.414952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.414713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:34.727177 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.727148    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:10:34.728049 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:34.728024    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0603f66a_45f8_47ee_8f72_ff05bf144855.slice/crio-4b694627bc61ce89db19d380d3d6cc0cf923708acf327d17ed3eeb93d72bed0f WatchSource:0}: Error finding container 4b694627bc61ce89db19d380d3d6cc0cf923708acf327d17ed3eeb93d72bed0f: Status 404 returned error can't find the container with id 4b694627bc61ce89db19d380d3d6cc0cf923708acf327d17ed3eeb93d72bed0f
Sep 15 21:10:34.761954 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:34.761928    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:10:34.766851 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:34.766727    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2e47ba_b5e4_4f31_b6ae_1b71c25a5289.slice/crio-75898bc381c2503d76dbe29716effc451ff697cfb5d50e7e4f9f61a9104f0eb5 WatchSource:0}: Error finding container 75898bc381c2503d76dbe29716effc451ff697cfb5d50e7e4f9f61a9104f0eb5: Status 404 returned error can't find the container with id 75898bc381c2503d76dbe29716effc451ff697cfb5d50e7e4f9f61a9104f0eb5
Sep 15 21:10:35.258652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.258611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" event={"ID":"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289","Type":"ContainerStarted","Data":"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"}
Sep 15 21:10:35.259072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.258684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" event={"ID":"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289","Type":"ContainerStarted","Data":"75898bc381c2503d76dbe29716effc451ff697cfb5d50e7e4f9f61a9104f0eb5"}
Sep 15 21:10:35.259072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.258762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:35.260854 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.260823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" event={"ID":"0603f66a-45f8-47ee-8f72-ff05bf144855","Type":"ContainerStarted","Data":"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"}
Sep 15 21:10:35.260973 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.260878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" event={"ID":"0603f66a-45f8-47ee-8f72-ff05bf144855","Type":"ContainerStarted","Data":"4b694627bc61ce89db19d380d3d6cc0cf923708acf327d17ed3eeb93d72bed0f"}
Sep 15 21:10:35.261418 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.261399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:35.295744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.295680    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" podStartSLOduration=1.295657839 podStartE2EDuration="1.295657839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:35.290832598 +0000 UTC m=+743.077200609" watchObservedRunningTime="2026-09-15 21:10:35.295657839 +0000 UTC m=+743.082025851"
Sep 15 21:10:35.331391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:35.331281    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" podStartSLOduration=1.331258893 podStartE2EDuration="1.331258893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:35.328907123 +0000 UTC m=+743.115275131" watchObservedRunningTime="2026-09-15 21:10:35.331258893 +0000 UTC m=+743.117626901"
Sep 15 21:10:36.185610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:36.185212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:10:36.185610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:36.185603    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:10:36.188741 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:36.188705    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 21:10:36.188901 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:36.188760    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 21:10:39.303127 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.303088    2577 generic.go:356] "Generic (PLEG): container finished" podID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerID="db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6" exitCode=0
Sep 15 21:10:39.303663 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.303168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerDied","Data":"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"}
Sep 15 21:10:39.303834 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.303816    2577 scope.go:117] "RemoveContainer" containerID="db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"
Sep 15 21:10:39.305001 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.304981    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c7a38b-af66-4a5e-9180-2451858939da" containerID="5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c" exitCode=0
Sep 15 21:10:39.305259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.305064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerDied","Data":"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"}
Sep 15 21:10:39.305259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.305107    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:39.305259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.305192    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:39.305431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.305285    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:39.305511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.305494    2577 scope.go:117] "RemoveContainer" containerID="5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"
Sep 15 21:10:39.306596 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.306532    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:39.306708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.306629    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:39.306708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:39.306677    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:40.312455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.312421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerStarted","Data":"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"}
Sep 15 21:10:40.314511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.314489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerStarted","Data":"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"}
Sep 15 21:10:40.578754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.578659    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:40.578984 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.578956    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:40.579068 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.579020    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:40.674346 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.674310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:40.674998 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.674970    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:40.675190 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:40.675169    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:41.579804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:41.579763    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:41.580337 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:41.579838    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:41.674890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:41.674851    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:41.675163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:41.675135    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:42.579007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:42.578958    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:42.579228 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:42.579050    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:42.674804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:42.674334    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:42.674804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:42.674399    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:43.578902 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:43.578868    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:43.579219 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:43.579175    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:43.675068 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:43.675024    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:43.675533 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:43.675094    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:44.318015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.316904    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:10:44.324408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.324384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.326210 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.326189    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:10:44.328102 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.327536    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-253ca1\""
Sep 15 21:10:44.328102 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.327830    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-253ca1-dockercfg-vbgvr\""
Sep 15 21:10:44.445602 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.445135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.445602 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.445209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4759\" (UniqueName: \"kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.548239 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.548176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.548553 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.548519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4759\" (UniqueName: \"kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.550007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.549974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.563455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.563428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4759\" (UniqueName: \"kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759\") pod \"payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:44.579818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.579742    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:44.580052 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.580023    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:44.675008 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.674975    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:44.675260 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.675230    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:44.691496 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:44.691461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:45.126537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.123253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:10:45.374239 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.374147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" event={"ID":"9bcce461-5117-438b-81c1-f93e7bc188d1","Type":"ContainerStarted","Data":"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"}
Sep 15 21:10:45.374496 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.374474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" event={"ID":"9bcce461-5117-438b-81c1-f93e7bc188d1","Type":"ContainerStarted","Data":"02a0f65edc8df2dfc02760b8332d6c834036c8fa6e6623fbfd163ee7443ddf87"}
Sep 15 21:10:45.375452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.375430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:45.394293 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.394221    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" podStartSLOduration=1.394183645 podStartE2EDuration="1.394183645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:45.392321057 +0000 UTC m=+753.178689067" watchObservedRunningTime="2026-09-15 21:10:45.394183645 +0000 UTC m=+753.180551657"
Sep 15 21:10:45.579076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.579038    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 15 21:10:45.579374 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.579320    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 15 21:10:45.676471 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.676372    2577 patch_prober.go:28] interesting pod/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 15 21:10:45.676471 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:45.676440    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 15 21:10:46.195591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.195490    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:10:46.199327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.199306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.204426 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.204398    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-05731f-dockercfg-hln5t\""
Sep 15 21:10:46.208424 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.208395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-05731f\""
Sep 15 21:10:46.209348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.209322    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:10:46.219207 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.214048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:10:46.219207 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.214144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.252043 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.252016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:10:46.294743 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.294703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:10:46.310533 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.310487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.310817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.310786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fp9b\" (UniqueName: \"kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.310978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.310962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.311604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.311583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nsqs\" (UniqueName: \"kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.412684 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.412651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nsqs\" (UniqueName: \"kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.412824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.412766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.412824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.412787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fp9b\" (UniqueName: \"kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.412824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.412806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.413460 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.413439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.413975 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.413952    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.422621 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.422604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nsqs\" (UniqueName: \"kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs\") pod \"payload-processing-e2e-autores-05731f-6b676db9b5-9745g\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") " pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.423283 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.423266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fp9b\" (UniqueName: \"kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b\") pod \"payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.527989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.527959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:46.553717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.553287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:46.585342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.585000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:46.586499 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.586358    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:46.587353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.587270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:10:46.748434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.746894    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:46.748434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.748258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:46.761203 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.761161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:10:46.866647 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.863414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:10:46.913801 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:46.913759    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:10:46.915370 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:46.915345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408e6840_1e4d_4959_9e5f_1e1c24a60ace.slice/crio-6a53cf21fb45485c0316547b3116ee306fbe9b930fff037973cf5c62cbea0b80 WatchSource:0}: Error finding container 6a53cf21fb45485c0316547b3116ee306fbe9b930fff037973cf5c62cbea0b80: Status 404 returned error can't find the container with id 6a53cf21fb45485c0316547b3116ee306fbe9b930fff037973cf5c62cbea0b80
Sep 15 21:10:47.293735 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.293705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:10:47.405768 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.405732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" event={"ID":"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4","Type":"ContainerStarted","Data":"64302bfe973bf10fd62c9656de28143fc6f2531f5d187241e82a1975787ba1e7"}
Sep 15 21:10:47.405915 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.405803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" event={"ID":"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4","Type":"ContainerStarted","Data":"241229ee4bcdd44242f643b815651b8036e7d0c44d70d776e16ba4d514658694"}
Sep 15 21:10:47.405977 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.405937    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:47.408370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.408347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" event={"ID":"408e6840-1e4d-4959-9e5f-1e1c24a60ace","Type":"ContainerStarted","Data":"7150f1279ec97c3812ebdbc543b28bc41bee6728b98806a39d048656c5660263"}
Sep 15 21:10:47.408503 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.408381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" event={"ID":"408e6840-1e4d-4959-9e5f-1e1c24a60ace","Type":"ContainerStarted","Data":"6a53cf21fb45485c0316547b3116ee306fbe9b930fff037973cf5c62cbea0b80"}
Sep 15 21:10:47.409177 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.409124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:10:47.486592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.485795    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" podStartSLOduration=1.485772256 podStartE2EDuration="1.485772256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:47.443551702 +0000 UTC m=+755.229919710" watchObservedRunningTime="2026-09-15 21:10:47.485772256 +0000 UTC m=+755.272140267"
Sep 15 21:10:47.490885 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:47.490823    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" podStartSLOduration=1.490798165 podStartE2EDuration="1.490798165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:47.484998321 +0000 UTC m=+755.271366357" watchObservedRunningTime="2026-09-15 21:10:47.490798165 +0000 UTC m=+755.277166183"
Sep 15 21:10:49.995604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:49.995557    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"]
Sep 15 21:10:50.000273 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.000254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:50.003769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.003751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:10:50.004254 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.004238    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:10:50.010859 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.010828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"]
Sep 15 21:10:50.056575 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.056535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh\" (UID: \"cd860ce5-1a9c-4d0d-8588-1f58bb816f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:50.157512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.157476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh\" (UID: \"cd860ce5-1a9c-4d0d-8588-1f58bb816f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:50.158104 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.158086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh\" (UID: \"cd860ce5-1a9c-4d0d-8588-1f58bb816f24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:50.312364 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.312275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:50.597685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:50.597657    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"]
Sep 15 21:10:50.599977 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:10:50.599951    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd860ce5_1a9c_4d0d_8588_1f58bb816f24.slice/crio-2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667 WatchSource:0}: Error finding container 2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667: Status 404 returned error can't find the container with id 2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667
Sep 15 21:10:51.435270 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:51.435233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh" event={"ID":"cd860ce5-1a9c-4d0d-8588-1f58bb816f24","Type":"ContainerStarted","Data":"2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667"}
Sep 15 21:10:52.443643 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.443614    2577 generic.go:356] "Generic (PLEG): container finished" podID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerID="78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292" exitCode=0
Sep 15 21:10:52.444105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.443692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerDied","Data":"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"}
Sep 15 21:10:52.444167 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.444138    2577 scope.go:117] "RemoveContainer" containerID="78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"
Sep 15 21:10:52.849459 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.849406    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:52.850307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.850277    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:52.850512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:52.850489    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:10:53.450981 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:53.450898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh" event={"ID":"cd860ce5-1a9c-4d0d-8588-1f58bb816f24","Type":"ContainerStarted","Data":"be06acadaa8cb698f543f975e3b9b9c574b8edf537c690324722a12ff29af40d"}
Sep 15 21:10:53.452918 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:53.452892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerStarted","Data":"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"}
Sep 15 21:10:53.467650 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:53.467607    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh" podStartSLOduration=2.184251357 podStartE2EDuration="4.46759274s" totalImagesPullingTime="2.283341383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:53.46576886 +0000 UTC m=+761.252136868" watchObservedRunningTime="2026-09-15 21:10:53.46759274 +0000 UTC m=+761.253960747"
Sep 15 21:10:54.002108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:54.002078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:54.002275 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:54.002244    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:54.002339 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:54.002318    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:54.465704 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:54.465655    2577 generic.go:356] "Generic (PLEG): container finished" podID="cd860ce5-1a9c-4d0d-8588-1f58bb816f24" containerID="be06acadaa8cb698f543f975e3b9b9c574b8edf537c690324722a12ff29af40d" exitCode=0
Sep 15 21:10:54.466408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:54.466377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh" event={"ID":"cd860ce5-1a9c-4d0d-8588-1f58bb816f24","Type":"ContainerDied","Data":"be06acadaa8cb698f543f975e3b9b9c574b8edf537c690324722a12ff29af40d"}
Sep 15 21:10:55.001962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.001921    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:55.002151 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.001977    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:55.659690 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.659614    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:55.716227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.716201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca\") pod \"cd860ce5-1a9c-4d0d-8588-1f58bb816f24\" (UID: \"cd860ce5-1a9c-4d0d-8588-1f58bb816f24\") "
Sep 15 21:10:55.716678 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.716658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd860ce5-1a9c-4d0d-8588-1f58bb816f24" (UID: "cd860ce5-1a9c-4d0d-8588-1f58bb816f24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:10:55.817864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:55.817753    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd860ce5-1a9c-4d0d-8588-1f58bb816f24-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:10:56.002073 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:56.002035    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:56.002250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:56.002090    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:56.479514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:56.479474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh" event={"ID":"cd860ce5-1a9c-4d0d-8588-1f58bb816f24","Type":"ContainerDied","Data":"2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667"}
Sep 15 21:10:56.479714 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:56.479527    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e2eda7526aff64cb38bfe3ecdfb6e687f2a0fff9d214948ad8b29be9e4ab667"
Sep 15 21:10:56.479714 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:56.479489    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"
Sep 15 21:10:57.002094 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:57.002052    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:57.002839 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:57.002807    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:57.410681 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:57.410600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:10:58.002242 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:58.002211    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:58.002650 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:58.002263    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:58.416419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:58.416327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:10:59.002153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:59.002118    2577 patch_prober.go:28] interesting pod/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 15 21:10:59.002333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:59.002174    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 15 21:10:59.203315 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:59.203274    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:10:59.421868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:10:59.421780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:11:00.010683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:00.010639    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:11:00.012180 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:00.012154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:11:03.812263 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:03.812230    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:11:03.812731 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:03.812495    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" podUID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" containerName="payload-processing" containerID="cri-o://2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6" gracePeriod=30
Sep 15 21:11:03.822333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:03.822308    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:11:03.822593 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:03.822522    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" podUID="0603f66a-45f8-47ee-8f72-ff05bf144855" containerName="payload-pre-processing" containerID="cri-o://52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2" gracePeriod=30
Sep 15 21:11:04.099598 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.099525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"]
Sep 15 21:11:04.103152 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.103129    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bfe4d22a-9bsfh"]
Sep 15 21:11:04.111224 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.111195    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:11:04.114408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.114384    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:11:04.179684 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.178051    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:11:04.179684 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.178297    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy" containerID="cri-o://632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38" gracePeriod=30
Sep 15 21:11:04.214791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.214763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4hs6\" (UniqueName: \"kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6\") pod \"0603f66a-45f8-47ee-8f72-ff05bf144855\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") "
Sep 15 21:11:04.214980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.214807    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume\") pod \"0603f66a-45f8-47ee-8f72-ff05bf144855\" (UID: \"0603f66a-45f8-47ee-8f72-ff05bf144855\") "
Sep 15 21:11:04.214980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.214932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-876bm\" (UniqueName: \"kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm\") pod \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") "
Sep 15 21:11:04.215116 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.215048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume\") pod \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\" (UID: \"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289\") "
Sep 15 21:11:04.215227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.215201    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0603f66a-45f8-47ee-8f72-ff05bf144855" (UID: "0603f66a-45f8-47ee-8f72-ff05bf144855"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:04.215513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.215486    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0603f66a-45f8-47ee-8f72-ff05bf144855-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:04.215611 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.215536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" (UID: "8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:04.217239 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.217219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6" (OuterVolumeSpecName: "kube-api-access-n4hs6") pod "0603f66a-45f8-47ee-8f72-ff05bf144855" (UID: "0603f66a-45f8-47ee-8f72-ff05bf144855"). InnerVolumeSpecName "kube-api-access-n4hs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:04.217301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.217243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm" (OuterVolumeSpecName: "kube-api-access-876bm") pod "8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" (UID: "8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289"). InnerVolumeSpecName "kube-api-access-876bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:04.316321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.316278    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:04.316321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.316327    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4hs6\" (UniqueName: \"kubernetes.io/projected/0603f66a-45f8-47ee-8f72-ff05bf144855-kube-api-access-n4hs6\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:04.316628 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.316342    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-876bm\" (UniqueName: \"kubernetes.io/projected/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289-kube-api-access-876bm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:04.525905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.525871    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" containerID="2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6" exitCode=0
Sep 15 21:11:04.526109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.525932    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"
Sep 15 21:11:04.526109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.525962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" event={"ID":"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289","Type":"ContainerDied","Data":"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"}
Sep 15 21:11:04.526109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.526019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5" event={"ID":"8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289","Type":"ContainerDied","Data":"75898bc381c2503d76dbe29716effc451ff697cfb5d50e7e4f9f61a9104f0eb5"}
Sep 15 21:11:04.526109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.526049    2577 scope.go:117] "RemoveContainer" containerID="2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"
Sep 15 21:11:04.527682 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.527657    2577 generic.go:356] "Generic (PLEG): container finished" podID="0603f66a-45f8-47ee-8f72-ff05bf144855" containerID="52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2" exitCode=0
Sep 15 21:11:04.527761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.527703    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"
Sep 15 21:11:04.527761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.527742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" event={"ID":"0603f66a-45f8-47ee-8f72-ff05bf144855","Type":"ContainerDied","Data":"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"}
Sep 15 21:11:04.527881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.527786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq" event={"ID":"0603f66a-45f8-47ee-8f72-ff05bf144855","Type":"ContainerDied","Data":"4b694627bc61ce89db19d380d3d6cc0cf923708acf327d17ed3eeb93d72bed0f"}
Sep 15 21:11:04.536958 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.536937    2577 scope.go:117] "RemoveContainer" containerID="2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"
Sep 15 21:11:04.537391 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:04.537364    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6\": container with ID starting with 2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6 not found: ID does not exist" containerID="2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"
Sep 15 21:11:04.537514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.537401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6"} err="failed to get container status \"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6\": rpc error: code = NotFound desc = could not find container \"2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6\": container with ID starting with 2022f1b9cf9879f6d82438d51520e8a013657569a0de53aaee7e30607ee526e6 not found: ID does not exist"
Sep 15 21:11:04.537514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.537423    2577 scope.go:117] "RemoveContainer" containerID="52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"
Sep 15 21:11:04.546097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.546080    2577 scope.go:117] "RemoveContainer" containerID="52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"
Sep 15 21:11:04.546396 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:04.546381    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2\": container with ID starting with 52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2 not found: ID does not exist" containerID="52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"
Sep 15 21:11:04.546463 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.546404    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2"} err="failed to get container status \"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2\": rpc error: code = NotFound desc = could not find container \"52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2\": container with ID starting with 52c3d2965329d09827acfa6b81d6ffcba0d10b00d4df2c8859790124b2069ef2 not found: ID does not exist"
Sep 15 21:11:04.551952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.551828    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:11:04.554710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.554689    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bfe4d22a-59799d5d8c-84rr5"]
Sep 15 21:11:04.566136 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.566115    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:11:04.570935 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.570913    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bfe4d22a-54cbbf8ffd-8k9fq"]
Sep 15 21:11:04.696962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.696931    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0603f66a-45f8-47ee-8f72-ff05bf144855" path="/var/lib/kubelet/pods/0603f66a-45f8-47ee-8f72-ff05bf144855/volumes"
Sep 15 21:11:04.697344 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.697330    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" path="/var/lib/kubelet/pods/8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289/volumes"
Sep 15 21:11:04.697760 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:04.697740    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd860ce5-1a9c-4d0d-8588-1f58bb816f24" path="/var/lib/kubelet/pods/cd860ce5-1a9c-4d0d-8588-1f58bb816f24/volumes"
Sep 15 21:11:09.533749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.533724    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:11:09.563399 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.563326    2577 generic.go:356] "Generic (PLEG): container finished" podID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerID="632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38" exitCode=0
Sep 15 21:11:09.563619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.563490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerDied","Data":"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"}
Sep 15 21:11:09.563619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.563540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv" event={"ID":"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96","Type":"ContainerDied","Data":"dccfba1c8950d5ec74c8c0b727d7a1ebfe044edc5ce23402558ae8eb7fb3d31e"}
Sep 15 21:11:09.563619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.563604    2577 scope.go:117] "RemoveContainer" containerID="632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"
Sep 15 21:11:09.563777 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.563764    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"
Sep 15 21:11:09.575486 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.575463    2577 scope.go:117] "RemoveContainer" containerID="78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"
Sep 15 21:11:09.584413 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.584391    2577 scope.go:117] "RemoveContainer" containerID="632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"
Sep 15 21:11:09.584800 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:09.584764    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38\": container with ID starting with 632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38 not found: ID does not exist" containerID="632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"
Sep 15 21:11:09.584889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.584810    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38"} err="failed to get container status \"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38\": rpc error: code = NotFound desc = could not find container \"632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38\": container with ID starting with 632728260f5b84a61672d4207ceeacf4cdedc3b7981335fd95c58c041266bc38 not found: ID does not exist"
Sep 15 21:11:09.584889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.584829    2577 scope.go:117] "RemoveContainer" containerID="78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"
Sep 15 21:11:09.585127 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:09.585106    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292\": container with ID starting with 78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292 not found: ID does not exist" containerID="78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"
Sep 15 21:11:09.585203 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.585138    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292"} err="failed to get container status \"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292\": rpc error: code = NotFound desc = could not find container \"78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292\": container with ID starting with 78afff4c59afb390f0912360065771cdbce84bf6c3938eb23e3b14445eaf9292 not found: ID does not exist"
Sep 15 21:11:09.675455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675376    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675434    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phvrh\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675696    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675729    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.675786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:11:09.676050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:11:09.676050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.676050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675856    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs\") pod \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\" (UID: \"d1aa8f99-cd0b-44c4-8a70-56da82eb2f96\") "
Sep 15 21:11:09.676050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.675946    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:11:09.676050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676005    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:09.676392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:11:09.676648 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676628    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.676699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676656    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.676699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676670    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.676699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676683    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.676699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.676697    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.677757 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.677736    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:11:09.677980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.677958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh" (OuterVolumeSpecName: "kube-api-access-phvrh") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "kube-api-access-phvrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:09.678431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.678406    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:09.678431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.678403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" (UID: "d1aa8f99-cd0b-44c4-8a70-56da82eb2f96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:11:09.777792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.777756    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.777792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.777786    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.777792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.777797    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.778038 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.777808    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phvrh\" (UniqueName: \"kubernetes.io/projected/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96-kube-api-access-phvrh\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:09.887701 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.887667    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:11:09.893394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:09.893368    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bfe4d22a-openshift-default-699f4c97f6-7qmkv"]
Sep 15 21:11:10.691358 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:10.691320    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" path="/var/lib/kubelet/pods/d1aa8f99-cd0b-44c4-8a70-56da82eb2f96/volumes"
Sep 15 21:11:11.645543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.645507    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:11.646263 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646243    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0603f66a-45f8-47ee-8f72-ff05bf144855" containerName="payload-pre-processing"
Sep 15 21:11:11.646263 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646265    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0603f66a-45f8-47ee-8f72-ff05bf144855" containerName="payload-pre-processing"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646283    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd860ce5-1a9c-4d0d-8588-1f58bb816f24" containerName="revoke-keys"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646288    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd860ce5-1a9c-4d0d-8588-1f58bb816f24" containerName="revoke-keys"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646302    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646308    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646314    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" containerName="payload-processing"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646320    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" containerName="payload-processing"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646333    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646338    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646398    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0603f66a-45f8-47ee-8f72-ff05bf144855" containerName="payload-pre-processing"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646404    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd860ce5-1a9c-4d0d-8588-1f58bb816f24" containerName="revoke-keys"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646413    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d2e47ba-b5e4-4f31-b6ae-1b71c25a5289" containerName="payload-processing"
Sep 15 21:11:11.646415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646419    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.646850 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.646426    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1aa8f99-cd0b-44c4-8a70-56da82eb2f96" containerName="istio-proxy"
Sep 15 21:11:11.650867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.650850    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.653496 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.653472    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-6529f90b\""
Sep 15 21:11:11.660178 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.660153    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:11.662663 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.662639    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:11.666644 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.666424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.669076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.669056    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-6529f90b-dockercfg-2jzgk\""
Sep 15 21:11:11.674310 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.674289    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:11.799236 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.799192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqn5b\" (UniqueName: \"kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.799755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.799262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjh5\" (UniqueName: \"kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.799755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.799371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.799755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.799396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.900231 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.900231 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.900534 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqn5b\" (UniqueName: \"kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.900534 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjh5\" (UniqueName: \"kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.900899 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.900997 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.900981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.909365 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.909342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjh5\" (UniqueName: \"kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5\") pod \"payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.909475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.909390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqn5b\" (UniqueName: \"kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b\") pod \"payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") " pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:11.963363 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.963333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:11.979550 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:11.979525    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:12.241287 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.241106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:12.243608 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:11:12.243555    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5d367a_78cb_4368_aa20_f6e7ad58da89.slice/crio-5c94ee4c376a379575a619ab4ef139f359a9601030a28b5217bff951bbbea34d WatchSource:0}: Error finding container 5c94ee4c376a379575a619ab4ef139f359a9601030a28b5217bff951bbbea34d: Status 404 returned error can't find the container with id 5c94ee4c376a379575a619ab4ef139f359a9601030a28b5217bff951bbbea34d
Sep 15 21:11:12.267491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.267170    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:12.277744 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:11:12.277715    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098cd26c_b0ea_4a29_b122_36a64832ca7c.slice/crio-455535c100d946c404cefbeed5e757cfb523ed43a58b13283c9a3421c9a75252 WatchSource:0}: Error finding container 455535c100d946c404cefbeed5e757cfb523ed43a58b13283c9a3421c9a75252: Status 404 returned error can't find the container with id 455535c100d946c404cefbeed5e757cfb523ed43a58b13283c9a3421c9a75252
Sep 15 21:11:12.586757 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.586660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" event={"ID":"098cd26c-b0ea-4a29-b122-36a64832ca7c","Type":"ContainerStarted","Data":"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"}
Sep 15 21:11:12.586757 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.586723    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:12.586757 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.586741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" event={"ID":"098cd26c-b0ea-4a29-b122-36a64832ca7c","Type":"ContainerStarted","Data":"455535c100d946c404cefbeed5e757cfb523ed43a58b13283c9a3421c9a75252"}
Sep 15 21:11:12.588225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.588199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" event={"ID":"cf5d367a-78cb-4368-aa20-f6e7ad58da89","Type":"ContainerStarted","Data":"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"}
Sep 15 21:11:12.588443 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.588260    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:12.588443 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.588283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" event={"ID":"cf5d367a-78cb-4368-aa20-f6e7ad58da89","Type":"ContainerStarted","Data":"5c94ee4c376a379575a619ab4ef139f359a9601030a28b5217bff951bbbea34d"}
Sep 15 21:11:12.606729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.606666    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" podStartSLOduration=1.60664579 podStartE2EDuration="1.60664579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:12.604100025 +0000 UTC m=+780.390468033" watchObservedRunningTime="2026-09-15 21:11:12.60664579 +0000 UTC m=+780.393013801"
Sep 15 21:11:12.621839 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:12.621790    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" podStartSLOduration=1.621772786 podStartE2EDuration="1.621772786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:12.619233179 +0000 UTC m=+780.405601187" watchObservedRunningTime="2026-09-15 21:11:12.621772786 +0000 UTC m=+780.408140792"
Sep 15 21:11:23.597779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:23.597742    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:23.598270 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:23.597834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:37.551524 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.551443    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"]
Sep 15 21:11:37.555085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.555070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:37.557925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.557893    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:11:37.558069 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.557952    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:11:37.565390 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.565355    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"]
Sep 15 21:11:37.649077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.648847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs\" (UID: \"b594c166-de8e-4ac6-b1da-81022871e2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:37.750138 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.750106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs\" (UID: \"b594c166-de8e-4ac6-b1da-81022871e2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:37.750731 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.750712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs\" (UID: \"b594c166-de8e-4ac6-b1da-81022871e2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:37.871006 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:37.870920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:38.145032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:38.145008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"]
Sep 15 21:11:38.148080 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:11:38.148052    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb594c166_de8e_4ac6_b1da_81022871e2a8.slice/crio-9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22 WatchSource:0}: Error finding container 9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22: Status 404 returned error can't find the container with id 9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22
Sep 15 21:11:38.750644 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:38.750607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs" event={"ID":"b594c166-de8e-4ac6-b1da-81022871e2a8","Type":"ContainerStarted","Data":"463a874757d814c9031b245a247f565df3ec0e83b3e5aa5cad28177605759df9"}
Sep 15 21:11:38.751017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:38.750662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs" event={"ID":"b594c166-de8e-4ac6-b1da-81022871e2a8","Type":"ContainerStarted","Data":"9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22"}
Sep 15 21:11:38.766886 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:38.766834    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs" podStartSLOduration=1.766819114 podStartE2EDuration="1.766819114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:38.765288251 +0000 UTC m=+806.551656260" watchObservedRunningTime="2026-09-15 21:11:38.766819114 +0000 UTC m=+806.553187137"
Sep 15 21:11:39.758482 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:39.758440    2577 generic.go:356] "Generic (PLEG): container finished" podID="b594c166-de8e-4ac6-b1da-81022871e2a8" containerID="463a874757d814c9031b245a247f565df3ec0e83b3e5aa5cad28177605759df9" exitCode=0
Sep 15 21:11:39.758912 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:39.758474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs" event={"ID":"b594c166-de8e-4ac6-b1da-81022871e2a8","Type":"ContainerDied","Data":"463a874757d814c9031b245a247f565df3ec0e83b3e5aa5cad28177605759df9"}
Sep 15 21:11:40.898315 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:40.898292    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:40.985609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:40.985560    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca\") pod \"b594c166-de8e-4ac6-b1da-81022871e2a8\" (UID: \"b594c166-de8e-4ac6-b1da-81022871e2a8\") "
Sep 15 21:11:40.985957 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:40.985935    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b594c166-de8e-4ac6-b1da-81022871e2a8" (UID: "b594c166-de8e-4ac6-b1da-81022871e2a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:40.986051 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:40.986037    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b594c166-de8e-4ac6-b1da-81022871e2a8-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:41.778693 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:41.778663    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"
Sep 15 21:11:41.778867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:41.778696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs" event={"ID":"b594c166-de8e-4ac6-b1da-81022871e2a8","Type":"ContainerDied","Data":"9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22"}
Sep 15 21:11:41.778867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:41.778765    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f5cb608674e6ffa1503ba269796c0aed338a19694b0f60b3402681ed3be3e22"
Sep 15 21:11:48.794385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.794352    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:48.794997 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.794588    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" podUID="098cd26c-b0ea-4a29-b122-36a64832ca7c" containerName="payload-processing" containerID="cri-o://6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed" gracePeriod=30
Sep 15 21:11:48.803726 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.803695    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:48.803986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.803938    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" podUID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" containerName="payload-pre-processing" containerID="cri-o://f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853" gracePeriod=30
Sep 15 21:11:48.966970 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.966905    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"]
Sep 15 21:11:48.969241 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:48.969217    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6529f90b-7tgcs"]
Sep 15 21:11:49.110380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.110354    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:49.112396 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.112378    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:49.271916 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.271827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume\") pod \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") "
Sep 15 21:11:49.272104 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.271989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkjh5\" (UniqueName: \"kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5\") pod \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\" (UID: \"cf5d367a-78cb-4368-aa20-f6e7ad58da89\") "
Sep 15 21:11:49.272104 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.272080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf5d367a-78cb-4368-aa20-f6e7ad58da89" (UID: "cf5d367a-78cb-4368-aa20-f6e7ad58da89"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:49.272229 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.272123    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqn5b\" (UniqueName: \"kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b\") pod \"098cd26c-b0ea-4a29-b122-36a64832ca7c\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") "
Sep 15 21:11:49.272229 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.272169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume\") pod \"098cd26c-b0ea-4a29-b122-36a64832ca7c\" (UID: \"098cd26c-b0ea-4a29-b122-36a64832ca7c\") "
Sep 15 21:11:49.272507 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.272479    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d367a-78cb-4368-aa20-f6e7ad58da89-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:49.272654 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.272525    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "098cd26c-b0ea-4a29-b122-36a64832ca7c" (UID: "098cd26c-b0ea-4a29-b122-36a64832ca7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:11:49.274238 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.274215    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5" (OuterVolumeSpecName: "kube-api-access-nkjh5") pod "cf5d367a-78cb-4368-aa20-f6e7ad58da89" (UID: "cf5d367a-78cb-4368-aa20-f6e7ad58da89"). InnerVolumeSpecName "kube-api-access-nkjh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:49.274321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.274246    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b" (OuterVolumeSpecName: "kube-api-access-xqn5b") pod "098cd26c-b0ea-4a29-b122-36a64832ca7c" (UID: "098cd26c-b0ea-4a29-b122-36a64832ca7c"). InnerVolumeSpecName "kube-api-access-xqn5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:11:49.373947 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.373866    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/098cd26c-b0ea-4a29-b122-36a64832ca7c-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:49.373947 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.373899    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkjh5\" (UniqueName: \"kubernetes.io/projected/cf5d367a-78cb-4368-aa20-f6e7ad58da89-kube-api-access-nkjh5\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:49.373947 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.373909    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqn5b\" (UniqueName: \"kubernetes.io/projected/098cd26c-b0ea-4a29-b122-36a64832ca7c-kube-api-access-xqn5b\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:11:49.826047 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.826018    2577 generic.go:356] "Generic (PLEG): container finished" podID="098cd26c-b0ea-4a29-b122-36a64832ca7c" containerID="6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed" exitCode=0
Sep 15 21:11:49.826454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.826063    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"
Sep 15 21:11:49.826454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.826106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" event={"ID":"098cd26c-b0ea-4a29-b122-36a64832ca7c","Type":"ContainerDied","Data":"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"}
Sep 15 21:11:49.826454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.826159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk" event={"ID":"098cd26c-b0ea-4a29-b122-36a64832ca7c","Type":"ContainerDied","Data":"455535c100d946c404cefbeed5e757cfb523ed43a58b13283c9a3421c9a75252"}
Sep 15 21:11:49.826454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.826188    2577 scope.go:117] "RemoveContainer" containerID="6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"
Sep 15 21:11:49.827689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.827670    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" containerID="f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853" exitCode=0
Sep 15 21:11:49.827770 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.827743    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"
Sep 15 21:11:49.827770 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.827753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" event={"ID":"cf5d367a-78cb-4368-aa20-f6e7ad58da89","Type":"ContainerDied","Data":"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"}
Sep 15 21:11:49.827861 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.827806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h" event={"ID":"cf5d367a-78cb-4368-aa20-f6e7ad58da89","Type":"ContainerDied","Data":"5c94ee4c376a379575a619ab4ef139f359a9601030a28b5217bff951bbbea34d"}
Sep 15 21:11:49.839927 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.839901    2577 scope.go:117] "RemoveContainer" containerID="6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"
Sep 15 21:11:49.840335 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:49.840306    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed\": container with ID starting with 6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed not found: ID does not exist" containerID="6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"
Sep 15 21:11:49.840444 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.840349    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed"} err="failed to get container status \"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed\": rpc error: code = NotFound desc = could not find container \"6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed\": container with ID starting with 6f1eb92655275219195a4b49e0d5b2e3889c65904ede5ba84ba9480cd0091fed not found: ID does not exist"
Sep 15 21:11:49.840444 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.840376    2577 scope.go:117] "RemoveContainer" containerID="f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"
Sep 15 21:11:49.850822 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.850754    2577 scope.go:117] "RemoveContainer" containerID="f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"
Sep 15 21:11:49.851531 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:11:49.851507    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853\": container with ID starting with f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853 not found: ID does not exist" containerID="f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"
Sep 15 21:11:49.851654 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.851541    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853"} err="failed to get container status \"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853\": rpc error: code = NotFound desc = could not find container \"f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853\": container with ID starting with f2a4de3febbce8574e3de294b6c79815a59f4705cb718a79de5bffeda7b13853 not found: ID does not exist"
Sep 15 21:11:49.859696 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.859673    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:49.862451 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.862425    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6529f90b-8476ffcbfd-bczzk"]
Sep 15 21:11:49.890324 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.890294    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:49.892342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:49.892317    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6529f90b-f857cbc4-zpz9h"]
Sep 15 21:11:50.690865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:50.690826    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="098cd26c-b0ea-4a29-b122-36a64832ca7c" path="/var/lib/kubelet/pods/098cd26c-b0ea-4a29-b122-36a64832ca7c/volumes"
Sep 15 21:11:50.691266 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:50.691245    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b594c166-de8e-4ac6-b1da-81022871e2a8" path="/var/lib/kubelet/pods/b594c166-de8e-4ac6-b1da-81022871e2a8/volumes"
Sep 15 21:11:50.691685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:50.691666    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" path="/var/lib/kubelet/pods/cf5d367a-78cb-4368-aa20-f6e7ad58da89/volumes"
Sep 15 21:11:55.445300 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445265    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:11:55.445770 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445756    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b594c166-de8e-4ac6-b1da-81022871e2a8" containerName="revoke-keys"
Sep 15 21:11:55.445817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445772    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b594c166-de8e-4ac6-b1da-81022871e2a8" containerName="revoke-keys"
Sep 15 21:11:55.445817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445785    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="098cd26c-b0ea-4a29-b122-36a64832ca7c" containerName="payload-processing"
Sep 15 21:11:55.445817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445792    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="098cd26c-b0ea-4a29-b122-36a64832ca7c" containerName="payload-processing"
Sep 15 21:11:55.445817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445811    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" containerName="payload-pre-processing"
Sep 15 21:11:55.445937 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445818    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" containerName="payload-pre-processing"
Sep 15 21:11:55.445937 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445884    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b594c166-de8e-4ac6-b1da-81022871e2a8" containerName="revoke-keys"
Sep 15 21:11:55.445937 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445897    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="098cd26c-b0ea-4a29-b122-36a64832ca7c" containerName="payload-processing"
Sep 15 21:11:55.445937 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.445907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5d367a-78cb-4368-aa20-f6e7ad58da89" containerName="payload-pre-processing"
Sep 15 21:11:55.456060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.456029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.457981 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.457952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:11:55.459104 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.459082    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-817c786c\""
Sep 15 21:11:55.469016 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.468986    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:11:55.474075 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.474052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.476924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.476904    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-817c786c-dockercfg-rqkt9\""
Sep 15 21:11:55.484913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.484890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:11:55.636301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.636266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2g4v\" (UniqueName: \"kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.636301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.636305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzc7s\" (UniqueName: \"kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.636599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.636334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.636599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.636350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.737348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.737310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2g4v\" (UniqueName: \"kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.737547 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.737367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzc7s\" (UniqueName: \"kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.737547 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.737419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.737547 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.737438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.738162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.738140    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.738230 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.738196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.747859 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.747834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2g4v\" (UniqueName: \"kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v\") pod \"payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") " pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:55.748470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.748452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzc7s\" (UniqueName: \"kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s\") pod \"payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.770318 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.770295    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:55.786527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:55.786505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:56.046234 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.046208    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:11:56.048218 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:11:56.048189    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66f6467_b8d2_4717_b6b7_837da5bb317f.slice/crio-6f89c354872f9ae52ba3e835d71bca24f872292968ee71290a1fde22b981a004 WatchSource:0}: Error finding container 6f89c354872f9ae52ba3e835d71bca24f872292968ee71290a1fde22b981a004: Status 404 returned error can't find the container with id 6f89c354872f9ae52ba3e835d71bca24f872292968ee71290a1fde22b981a004
Sep 15 21:11:56.091531 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.091508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:11:56.093178 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:11:56.093154    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4290348_5883_4302_a48e_4faeedade799.slice/crio-e9aa404265d18defca3b9f417f909e676fd7116883ca2a9577b87a34b91dce51 WatchSource:0}: Error finding container e9aa404265d18defca3b9f417f909e676fd7116883ca2a9577b87a34b91dce51: Status 404 returned error can't find the container with id e9aa404265d18defca3b9f417f909e676fd7116883ca2a9577b87a34b91dce51
Sep 15 21:11:56.882342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.882304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" event={"ID":"e4290348-5883-4302-a48e-4faeedade799","Type":"ContainerStarted","Data":"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"}
Sep 15 21:11:56.882803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.882370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" event={"ID":"e4290348-5883-4302-a48e-4faeedade799","Type":"ContainerStarted","Data":"e9aa404265d18defca3b9f417f909e676fd7116883ca2a9577b87a34b91dce51"}
Sep 15 21:11:56.882803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.882408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:11:56.884053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.884030    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" event={"ID":"f66f6467-b8d2-4717-b6b7-837da5bb317f","Type":"ContainerStarted","Data":"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"}
Sep 15 21:11:56.884140 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.884074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" event={"ID":"f66f6467-b8d2-4717-b6b7-837da5bb317f","Type":"ContainerStarted","Data":"6f89c354872f9ae52ba3e835d71bca24f872292968ee71290a1fde22b981a004"}
Sep 15 21:11:56.884179 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.884167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:11:56.901680 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.901636    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" podStartSLOduration=1.901620987 podStartE2EDuration="1.901620987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:56.898542723 +0000 UTC m=+824.684910731" watchObservedRunningTime="2026-09-15 21:11:56.901620987 +0000 UTC m=+824.687989057"
Sep 15 21:11:56.916344 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:11:56.916291    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" podStartSLOduration=1.916280964 podStartE2EDuration="1.916280964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:56.91610526 +0000 UTC m=+824.702473269" watchObservedRunningTime="2026-09-15 21:11:56.916280964 +0000 UTC m=+824.702648965"
Sep 15 21:12:07.893131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:07.893100    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:12:07.893623 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:07.893181    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:12:18.358927 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.358894    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"]
Sep 15 21:12:18.364157 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.364137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:18.366978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.366953    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:12:18.367232 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.367211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:12:18.371650 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.371626    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"]
Sep 15 21:12:18.467836 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.467800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-817c786c-4hslm\" (UID: \"68605d87-9b37-43e8-9525-72da56f95c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:18.568672 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.568633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-817c786c-4hslm\" (UID: \"68605d87-9b37-43e8-9525-72da56f95c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:18.569217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.569198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-817c786c-4hslm\" (UID: \"68605d87-9b37-43e8-9525-72da56f95c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:18.676981 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.676910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:18.955702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:18.955677    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"]
Sep 15 21:12:18.957238 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:12:18.957209    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68605d87_9b37_43e8_9525_72da56f95c7e.slice/crio-609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0 WatchSource:0}: Error finding container 609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0: Status 404 returned error can't find the container with id 609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0
Sep 15 21:12:19.008932 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:19.008905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm" event={"ID":"68605d87-9b37-43e8-9525-72da56f95c7e","Type":"ContainerStarted","Data":"609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0"}
Sep 15 21:12:20.019702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:20.019599    2577 generic.go:356] "Generic (PLEG): container finished" podID="68605d87-9b37-43e8-9525-72da56f95c7e" containerID="a538de9928f4e7c1b2674e9cef8b2077c91df11e74d9b550b3b24c04e7e7e076" exitCode=0
Sep 15 21:12:20.019702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:20.019675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm" event={"ID":"68605d87-9b37-43e8-9525-72da56f95c7e","Type":"ContainerDied","Data":"a538de9928f4e7c1b2674e9cef8b2077c91df11e74d9b550b3b24c04e7e7e076"}
Sep 15 21:12:21.169167 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:21.169145    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:21.296958 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:21.296880    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca\") pod \"68605d87-9b37-43e8-9525-72da56f95c7e\" (UID: \"68605d87-9b37-43e8-9525-72da56f95c7e\") "
Sep 15 21:12:21.297214 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:21.297192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68605d87-9b37-43e8-9525-72da56f95c7e" (UID: "68605d87-9b37-43e8-9525-72da56f95c7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:12:21.398149 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:21.398110    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68605d87-9b37-43e8-9525-72da56f95c7e-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:22.035153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:22.035114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm" event={"ID":"68605d87-9b37-43e8-9525-72da56f95c7e","Type":"ContainerDied","Data":"609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0"}
Sep 15 21:12:22.035153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:22.035153    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"
Sep 15 21:12:22.035408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:22.035173    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="609cfc436b4ade321a4639938587ac6aa8bb4dcc06dc6c48f63821abcae0c2e0"
Sep 15 21:12:24.401129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.401096    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:12:24.402030 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.401995    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" podUID="e4290348-5883-4302-a48e-4faeedade799" containerName="payload-processing" containerID="cri-o://186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b" gracePeriod=30
Sep 15 21:12:24.411543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.411178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:12:24.411543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.411428    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" podUID="f66f6467-b8d2-4717-b6b7-837da5bb317f" containerName="payload-pre-processing" containerID="cri-o://d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d" gracePeriod=30
Sep 15 21:12:24.573521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.573447    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"]
Sep 15 21:12:24.576554 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.576516    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-817c786c-4hslm"]
Sep 15 21:12:24.679719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.679696    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:12:24.691129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.691100    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68605d87-9b37-43e8-9525-72da56f95c7e" path="/var/lib/kubelet/pods/68605d87-9b37-43e8-9525-72da56f95c7e/volumes"
Sep 15 21:12:24.712627 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.712555    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:12:24.859219 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859188    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume\") pod \"e4290348-5883-4302-a48e-4faeedade799\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") "
Sep 15 21:12:24.859416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzc7s\" (UniqueName: \"kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s\") pod \"f66f6467-b8d2-4717-b6b7-837da5bb317f\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") "
Sep 15 21:12:24.859416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2g4v\" (UniqueName: \"kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v\") pod \"e4290348-5883-4302-a48e-4faeedade799\" (UID: \"e4290348-5883-4302-a48e-4faeedade799\") "
Sep 15 21:12:24.859416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume\") pod \"f66f6467-b8d2-4717-b6b7-837da5bb317f\" (UID: \"f66f6467-b8d2-4717-b6b7-837da5bb317f\") "
Sep 15 21:12:24.859607 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859547    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4290348-5883-4302-a48e-4faeedade799" (UID: "e4290348-5883-4302-a48e-4faeedade799"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:12:24.859751 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859730    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4290348-5883-4302-a48e-4faeedade799-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:24.859751 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.859733    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f66f6467-b8d2-4717-b6b7-837da5bb317f" (UID: "f66f6467-b8d2-4717-b6b7-837da5bb317f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:12:24.861575 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.861537    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s" (OuterVolumeSpecName: "kube-api-access-bzc7s") pod "f66f6467-b8d2-4717-b6b7-837da5bb317f" (UID: "f66f6467-b8d2-4717-b6b7-837da5bb317f"). InnerVolumeSpecName "kube-api-access-bzc7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:12:24.861575 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.861544    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v" (OuterVolumeSpecName: "kube-api-access-p2g4v") pod "e4290348-5883-4302-a48e-4faeedade799" (UID: "e4290348-5883-4302-a48e-4faeedade799"). InnerVolumeSpecName "kube-api-access-p2g4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:12:24.960873 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.960794    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66f6467-b8d2-4717-b6b7-837da5bb317f-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:24.960873 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.960820    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzc7s\" (UniqueName: \"kubernetes.io/projected/f66f6467-b8d2-4717-b6b7-837da5bb317f-kube-api-access-bzc7s\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:24.960873 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:24.960829    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2g4v\" (UniqueName: \"kubernetes.io/projected/e4290348-5883-4302-a48e-4faeedade799-kube-api-access-p2g4v\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:25.059170 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.059118    2577 generic.go:356] "Generic (PLEG): container finished" podID="f66f6467-b8d2-4717-b6b7-837da5bb317f" containerID="d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d" exitCode=0
Sep 15 21:12:25.059369 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.059234    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"
Sep 15 21:12:25.059549 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.059234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" event={"ID":"f66f6467-b8d2-4717-b6b7-837da5bb317f","Type":"ContainerDied","Data":"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"}
Sep 15 21:12:25.059670 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.059610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf" event={"ID":"f66f6467-b8d2-4717-b6b7-837da5bb317f","Type":"ContainerDied","Data":"6f89c354872f9ae52ba3e835d71bca24f872292968ee71290a1fde22b981a004"}
Sep 15 21:12:25.059670 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.059653    2577 scope.go:117] "RemoveContainer" containerID="d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"
Sep 15 21:12:25.061691 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.061663    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4290348-5883-4302-a48e-4faeedade799" containerID="186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b" exitCode=0
Sep 15 21:12:25.061796 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.061708    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"
Sep 15 21:12:25.061796 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.061711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" event={"ID":"e4290348-5883-4302-a48e-4faeedade799","Type":"ContainerDied","Data":"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"}
Sep 15 21:12:25.061903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.061813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb" event={"ID":"e4290348-5883-4302-a48e-4faeedade799","Type":"ContainerDied","Data":"e9aa404265d18defca3b9f417f909e676fd7116883ca2a9577b87a34b91dce51"}
Sep 15 21:12:25.069656 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.069616    2577 scope.go:117] "RemoveContainer" containerID="d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"
Sep 15 21:12:25.069961 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:12:25.069941    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d\": container with ID starting with d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d not found: ID does not exist" containerID="d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"
Sep 15 21:12:25.070021 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.069969    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d"} err="failed to get container status \"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d\": rpc error: code = NotFound desc = could not find container \"d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d\": container with ID starting with d1c7ed097a85e0368c20fae2a38e71fc09120e63da6e002f73925f62cb5b9c9d not found: ID does not exist"
Sep 15 21:12:25.070021 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.069986    2577 scope.go:117] "RemoveContainer" containerID="186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"
Sep 15 21:12:25.078039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.078023    2577 scope.go:117] "RemoveContainer" containerID="186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"
Sep 15 21:12:25.078296 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:12:25.078277    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b\": container with ID starting with 186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b not found: ID does not exist" containerID="186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"
Sep 15 21:12:25.078356 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.078306    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b"} err="failed to get container status \"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b\": rpc error: code = NotFound desc = could not find container \"186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b\": container with ID starting with 186bb4021638b8104d74a12b7a12f7da462b36a1c0a0ae9111af0669c457d36b not found: ID does not exist"
Sep 15 21:12:25.083130 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.083109    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:12:25.087223 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.087198    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-817c786c-5f9d85f5fd-ll9hf"]
Sep 15 21:12:25.097820 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.097796    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:12:25.106953 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:25.105026    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-817c786c-5fffd4c57c-qpkhb"]
Sep 15 21:12:26.692908 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:26.692858    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4290348-5883-4302-a48e-4faeedade799" path="/var/lib/kubelet/pods/e4290348-5883-4302-a48e-4faeedade799/volumes"
Sep 15 21:12:26.694068 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:26.694043    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66f6467-b8d2-4717-b6b7-837da5bb317f" path="/var/lib/kubelet/pods/f66f6467-b8d2-4717-b6b7-837da5bb317f/volumes"
Sep 15 21:12:31.085442 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085411    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:12:31.085904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66f6467-b8d2-4717-b6b7-837da5bb317f" containerName="payload-pre-processing"
Sep 15 21:12:31.085904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085905    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66f6467-b8d2-4717-b6b7-837da5bb317f" containerName="payload-pre-processing"
Sep 15 21:12:31.085978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085917    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4290348-5883-4302-a48e-4faeedade799" containerName="payload-processing"
Sep 15 21:12:31.085978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085923    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4290348-5883-4302-a48e-4faeedade799" containerName="payload-processing"
Sep 15 21:12:31.085978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085931    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68605d87-9b37-43e8-9525-72da56f95c7e" containerName="revoke-keys"
Sep 15 21:12:31.085978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.085937    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="68605d87-9b37-43e8-9525-72da56f95c7e" containerName="revoke-keys"
Sep 15 21:12:31.086105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.086012    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="68605d87-9b37-43e8-9525-72da56f95c7e" containerName="revoke-keys"
Sep 15 21:12:31.086105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.086023    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4290348-5883-4302-a48e-4faeedade799" containerName="payload-processing"
Sep 15 21:12:31.086105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.086034    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66f6467-b8d2-4717-b6b7-837da5bb317f" containerName="payload-pre-processing"
Sep 15 21:12:31.090808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.090792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.094032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.094010    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4194cf76\""
Sep 15 21:12:31.102477 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.102453    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:12:31.110225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.110195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:12:31.112559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.112514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.112688 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.112636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52qgt\" (UniqueName: \"kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.114358 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.114344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.117119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.117096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4194cf76-dockercfg-6v96n\""
Sep 15 21:12:31.123241 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.123221    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:12:31.214009 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.213969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.214009 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.214018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4zvp\" (UniqueName: \"kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.214267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.214069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.214267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.214111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52qgt\" (UniqueName: \"kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.215001 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.214976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.225307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.225087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52qgt\" (UniqueName: \"kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt\") pod \"payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.314799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.314763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.314799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.314803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4zvp\" (UniqueName: \"kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.315527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.315507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.329421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.329384    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4zvp\" (UniqueName: \"kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp\") pod \"payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") " pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.403358 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.403270    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:31.426011 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.425983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:31.686821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.686794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:12:31.687737 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:12:31.687709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8881cec1_ad5a_405f_a997_2126a945bef3.slice/crio-4844737e3e9d9680c53c13dc1bdbe2df575d1a86751f22957f52a061042884ea WatchSource:0}: Error finding container 4844737e3e9d9680c53c13dc1bdbe2df575d1a86751f22957f52a061042884ea: Status 404 returned error can't find the container with id 4844737e3e9d9680c53c13dc1bdbe2df575d1a86751f22957f52a061042884ea
Sep 15 21:12:31.713804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:31.713777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:12:31.728771 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:12:31.728730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84302f54_352a_4ca3_8505_3fc75da2dcdb.slice/crio-aedb50ea6812a025d0b274eade3256285d13132f5b3abbfa9dc5b57a87f18ada WatchSource:0}: Error finding container aedb50ea6812a025d0b274eade3256285d13132f5b3abbfa9dc5b57a87f18ada: Status 404 returned error can't find the container with id aedb50ea6812a025d0b274eade3256285d13132f5b3abbfa9dc5b57a87f18ada
Sep 15 21:12:32.103463 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.103415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" event={"ID":"8881cec1-ad5a-405f-a997-2126a945bef3","Type":"ContainerStarted","Data":"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"}
Sep 15 21:12:32.103922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.103480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" event={"ID":"8881cec1-ad5a-405f-a997-2126a945bef3","Type":"ContainerStarted","Data":"4844737e3e9d9680c53c13dc1bdbe2df575d1a86751f22957f52a061042884ea"}
Sep 15 21:12:32.103922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.103506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:32.105219 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.105196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" event={"ID":"84302f54-352a-4ca3-8505-3fc75da2dcdb","Type":"ContainerStarted","Data":"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"}
Sep 15 21:12:32.105338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.105236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" event={"ID":"84302f54-352a-4ca3-8505-3fc75da2dcdb","Type":"ContainerStarted","Data":"aedb50ea6812a025d0b274eade3256285d13132f5b3abbfa9dc5b57a87f18ada"}
Sep 15 21:12:32.105338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.105276    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:32.127556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.127510    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" podStartSLOduration=1.12749609 podStartE2EDuration="1.12749609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:12:32.125661045 +0000 UTC m=+859.912029057" watchObservedRunningTime="2026-09-15 21:12:32.12749609 +0000 UTC m=+859.913864097"
Sep 15 21:12:32.143905 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:32.143860    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" podStartSLOduration=1.143844848 podStartE2EDuration="1.143844848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:12:32.143064902 +0000 UTC m=+859.929432911" watchObservedRunningTime="2026-09-15 21:12:32.143844848 +0000 UTC m=+859.930212855"
Sep 15 21:12:43.114331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:43.114297    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:12:43.114846 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:43.114370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:12:53.630853 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.630803    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"]
Sep 15 21:12:53.635390 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.635367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"]
Sep 15 21:12:53.635706 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.635647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:53.642343 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.642317    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:12:53.642470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.642372    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:12:53.756036 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.756006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk\" (UID: \"aed857dc-5283-4704-b079-8fefe242c014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:53.857015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.856980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk\" (UID: \"aed857dc-5283-4704-b079-8fefe242c014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:53.857640 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.857622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk\" (UID: \"aed857dc-5283-4704-b079-8fefe242c014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:53.950110 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:53.950021    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:54.223074 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:54.223048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"]
Sep 15 21:12:54.225066 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:12:54.225040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed857dc_5283_4704_b079_8fefe242c014.slice/crio-e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4 WatchSource:0}: Error finding container e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4: Status 404 returned error can't find the container with id e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4
Sep 15 21:12:55.238444 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:55.238413    2577 generic.go:356] "Generic (PLEG): container finished" podID="aed857dc-5283-4704-b079-8fefe242c014" containerID="3407f5cbb885032b15ed94332aba45656d5208b1017c5cc195d5bd83a385eed6" exitCode=0
Sep 15 21:12:55.238895 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:55.238496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk" event={"ID":"aed857dc-5283-4704-b079-8fefe242c014","Type":"ContainerDied","Data":"3407f5cbb885032b15ed94332aba45656d5208b1017c5cc195d5bd83a385eed6"}
Sep 15 21:12:55.238895 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:55.238555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk" event={"ID":"aed857dc-5283-4704-b079-8fefe242c014","Type":"ContainerStarted","Data":"e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4"}
Sep 15 21:12:56.389843 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:56.389819    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:12:56.484139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:56.484108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca\") pod \"aed857dc-5283-4704-b079-8fefe242c014\" (UID: \"aed857dc-5283-4704-b079-8fefe242c014\") "
Sep 15 21:12:56.484528 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:56.484501    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aed857dc-5283-4704-b079-8fefe242c014" (UID: "aed857dc-5283-4704-b079-8fefe242c014"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:12:56.585993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:56.585899    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed857dc-5283-4704-b079-8fefe242c014-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:12:57.251604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:57.251575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk" event={"ID":"aed857dc-5283-4704-b079-8fefe242c014","Type":"ContainerDied","Data":"e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4"}
Sep 15 21:12:57.251754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:57.251626    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1d220bfc00c41200b862e2ec19d41d62fe2aa4c59806300eb03b386eaa41fd4"
Sep 15 21:12:57.251754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:12:57.251581    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"
Sep 15 21:13:00.810840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:00.810805    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:13:00.811384 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:00.811062    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" podUID="84302f54-352a-4ca3-8505-3fc75da2dcdb" containerName="payload-processing" containerID="cri-o://250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8" gracePeriod=30
Sep 15 21:13:00.819520 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:00.819491    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:13:00.821459 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:00.821404    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" podUID="8881cec1-ad5a-405f-a997-2126a945bef3" containerName="payload-pre-processing" containerID="cri-o://72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419" gracePeriod=30
Sep 15 21:13:00.999206 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:00.999133    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"]
Sep 15 21:13:01.002809 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.002762    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4194cf76-xbbdk"]
Sep 15 21:13:01.102292 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.102235    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:13:01.124874 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.124838    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:13:01.137794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.137767    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume\") pod \"84302f54-352a-4ca3-8505-3fc75da2dcdb\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") "
Sep 15 21:13:01.137947 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.137816    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume\") pod \"8881cec1-ad5a-405f-a997-2126a945bef3\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") "
Sep 15 21:13:01.137947 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.137841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4zvp\" (UniqueName: \"kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp\") pod \"84302f54-352a-4ca3-8505-3fc75da2dcdb\" (UID: \"84302f54-352a-4ca3-8505-3fc75da2dcdb\") "
Sep 15 21:13:01.138067 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.138009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52qgt\" (UniqueName: \"kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt\") pod \"8881cec1-ad5a-405f-a997-2126a945bef3\" (UID: \"8881cec1-ad5a-405f-a997-2126a945bef3\") "
Sep 15 21:13:01.138269 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.138240    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8881cec1-ad5a-405f-a997-2126a945bef3" (UID: "8881cec1-ad5a-405f-a997-2126a945bef3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:13:01.138448 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.138411    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8881cec1-ad5a-405f-a997-2126a945bef3-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:13:01.138959 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.138934    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84302f54-352a-4ca3-8505-3fc75da2dcdb" (UID: "84302f54-352a-4ca3-8505-3fc75da2dcdb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:13:01.141063 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.141030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp" (OuterVolumeSpecName: "kube-api-access-r4zvp") pod "84302f54-352a-4ca3-8505-3fc75da2dcdb" (UID: "84302f54-352a-4ca3-8505-3fc75da2dcdb"). InnerVolumeSpecName "kube-api-access-r4zvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:13:01.142263 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.142225    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt" (OuterVolumeSpecName: "kube-api-access-52qgt") pod "8881cec1-ad5a-405f-a997-2126a945bef3" (UID: "8881cec1-ad5a-405f-a997-2126a945bef3"). InnerVolumeSpecName "kube-api-access-52qgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:13:01.240314 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.240278    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84302f54-352a-4ca3-8505-3fc75da2dcdb-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:13:01.240518 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.240401    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4zvp\" (UniqueName: \"kubernetes.io/projected/84302f54-352a-4ca3-8505-3fc75da2dcdb-kube-api-access-r4zvp\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:13:01.240518 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.240422    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52qgt\" (UniqueName: \"kubernetes.io/projected/8881cec1-ad5a-405f-a997-2126a945bef3-kube-api-access-52qgt\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:13:01.277029 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.276999    2577 generic.go:356] "Generic (PLEG): container finished" podID="8881cec1-ad5a-405f-a997-2126a945bef3" containerID="72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419" exitCode=0
Sep 15 21:13:01.277245 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.277045    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"
Sep 15 21:13:01.277245 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.277077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" event={"ID":"8881cec1-ad5a-405f-a997-2126a945bef3","Type":"ContainerDied","Data":"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"}
Sep 15 21:13:01.277245 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.277129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp" event={"ID":"8881cec1-ad5a-405f-a997-2126a945bef3","Type":"ContainerDied","Data":"4844737e3e9d9680c53c13dc1bdbe2df575d1a86751f22957f52a061042884ea"}
Sep 15 21:13:01.277245 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.277150    2577 scope.go:117] "RemoveContainer" containerID="72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"
Sep 15 21:13:01.278943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.278824    2577 generic.go:356] "Generic (PLEG): container finished" podID="84302f54-352a-4ca3-8505-3fc75da2dcdb" containerID="250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8" exitCode=0
Sep 15 21:13:01.278943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.278866    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"
Sep 15 21:13:01.278943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.278876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" event={"ID":"84302f54-352a-4ca3-8505-3fc75da2dcdb","Type":"ContainerDied","Data":"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"}
Sep 15 21:13:01.278943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.278913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv" event={"ID":"84302f54-352a-4ca3-8505-3fc75da2dcdb","Type":"ContainerDied","Data":"aedb50ea6812a025d0b274eade3256285d13132f5b3abbfa9dc5b57a87f18ada"}
Sep 15 21:13:01.288880 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.288861    2577 scope.go:117] "RemoveContainer" containerID="72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"
Sep 15 21:13:01.289247 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:13:01.289156    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419\": container with ID starting with 72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419 not found: ID does not exist" containerID="72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"
Sep 15 21:13:01.289247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.289191    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419"} err="failed to get container status \"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419\": rpc error: code = NotFound desc = could not find container \"72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419\": container with ID starting with 72573c17f559102c7b863ec24fe18020d9914e9e3f1824fb6adabde7a7751419 not found: ID does not exist"
Sep 15 21:13:01.289247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.289218    2577 scope.go:117] "RemoveContainer" containerID="250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"
Sep 15 21:13:01.302145 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.302126    2577 scope.go:117] "RemoveContainer" containerID="250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"
Sep 15 21:13:01.302416 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:13:01.302398    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8\": container with ID starting with 250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8 not found: ID does not exist" containerID="250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"
Sep 15 21:13:01.302476 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.302421    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8"} err="failed to get container status \"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8\": rpc error: code = NotFound desc = could not find container \"250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8\": container with ID starting with 250dc63950ee0113579df7301707b594b3eacf256c4c72cd072cbb4babdd92a8 not found: ID does not exist"
Sep 15 21:13:01.303651 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.303607    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:13:01.306243 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.306221    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4194cf76-6d5564fd46-4gghp"]
Sep 15 21:13:01.316538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.316518    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:13:01.322348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:01.322328    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4194cf76-6f4cb98f69-jrrmv"]
Sep 15 21:13:02.690993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:02.690959    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84302f54-352a-4ca3-8505-3fc75da2dcdb" path="/var/lib/kubelet/pods/84302f54-352a-4ca3-8505-3fc75da2dcdb/volumes"
Sep 15 21:13:02.691362 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:02.691317    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8881cec1-ad5a-405f-a997-2126a945bef3" path="/var/lib/kubelet/pods/8881cec1-ad5a-405f-a997-2126a945bef3/volumes"
Sep 15 21:13:02.691633 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:02.691620    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed857dc-5283-4704-b079-8fefe242c014" path="/var/lib/kubelet/pods/aed857dc-5283-4704-b079-8fefe242c014/volumes"
Sep 15 21:13:12.662336 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:12.662293    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:13:12.663935 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:12.663910    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:13:48.086435 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086389    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:13:48.086934 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086920    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84302f54-352a-4ca3-8505-3fc75da2dcdb" containerName="payload-processing"
Sep 15 21:13:48.086934 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086935    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="84302f54-352a-4ca3-8505-3fc75da2dcdb" containerName="payload-processing"
Sep 15 21:13:48.087040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086949    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed857dc-5283-4704-b079-8fefe242c014" containerName="revoke-keys"
Sep 15 21:13:48.087040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086954    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed857dc-5283-4704-b079-8fefe242c014" containerName="revoke-keys"
Sep 15 21:13:48.087040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086971    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8881cec1-ad5a-405f-a997-2126a945bef3" containerName="payload-pre-processing"
Sep 15 21:13:48.087040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.086977    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8881cec1-ad5a-405f-a997-2126a945bef3" containerName="payload-pre-processing"
Sep 15 21:13:48.087040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.087040    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="84302f54-352a-4ca3-8505-3fc75da2dcdb" containerName="payload-processing"
Sep 15 21:13:48.087196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.087050    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8881cec1-ad5a-405f-a997-2126a945bef3" containerName="payload-pre-processing"
Sep 15 21:13:48.087196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.087056    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed857dc-5283-4704-b079-8fefe242c014" containerName="revoke-keys"
Sep 15 21:13:48.090592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.090539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.094803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.094778    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-dca193\"/\"openshift-service-ca.crt\""
Sep 15 21:13:48.094938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.094778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-dca193\"/\"default-dockercfg-9254x\""
Sep 15 21:13:48.094938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.094906    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-dca193\"/\"premium-w4-dca193-kserve-self-signed-certs\""
Sep 15 21:13:48.095066 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.094934    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-dca193\"/\"kube-root-ca.crt\""
Sep 15 21:13:48.101821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.101798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:13:48.213468 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hccqs\" (UniqueName: \"kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.213669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.213669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.213669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.213821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.213821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.213758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hccqs\" (UniqueName: \"kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314517    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314781 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.314986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.314967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.315073 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.315053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.317024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.317004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.317181 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.317165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.322980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.322961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hccqs\" (UniqueName: \"kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs\") pod \"premium-w4-dca193-kserve-56f9974745-j4sbz\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") " pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.402381 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.402296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:48.678348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:48.678311    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:13:48.680336 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:13:48.680308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4bca5ac_ab7a_49af_b308_e95fed00a51c.slice/crio-ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3 WatchSource:0}: Error finding container ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3: Status 404 returned error can't find the container with id ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3
Sep 15 21:13:49.554006 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:49.553976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" event={"ID":"c4bca5ac-ab7a-49af-b308-e95fed00a51c","Type":"ContainerStarted","Data":"0c42445a0abf2b95669837725e7a959c0d215363d091cda843bea3c4cb80c495"}
Sep 15 21:13:49.554382 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:49.554014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" event={"ID":"c4bca5ac-ab7a-49af-b308-e95fed00a51c","Type":"ContainerStarted","Data":"ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3"}
Sep 15 21:13:49.574789 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:49.574734    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" podStartSLOduration=1.574718747 podStartE2EDuration="1.574718747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:13:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:13:49.572670405 +0000 UTC m=+937.359038410" watchObservedRunningTime="2026-09-15 21:13:49.574718747 +0000 UTC m=+937.361086768"
Sep 15 21:13:58.402622 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:58.402585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:58.402622 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:58.402634    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:58.415408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:58.415383    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:13:58.619837 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:13:58.619808    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:14:00.871163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.871130    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"]
Sep 15 21:14:00.874900 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.874883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:00.877588 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.877550    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:14:00.877689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.877607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:14:00.882309 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.882287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"]
Sep 15 21:14:00.942710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:00.942682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-05731f-r2kmt\" (UID: \"56d3349f-0e77-4a1c-b6ef-fb6ef171589a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:01.043991 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:01.043953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-05731f-r2kmt\" (UID: \"56d3349f-0e77-4a1c-b6ef-fb6ef171589a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:01.044589 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:01.044543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-05731f-r2kmt\" (UID: \"56d3349f-0e77-4a1c-b6ef-fb6ef171589a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:01.188734 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:01.188651    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:01.466511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:01.466468    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"]
Sep 15 21:14:01.468088 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:14:01.468056    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d3349f_0e77_4a1c_b6ef_fb6ef171589a.slice/crio-9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99 WatchSource:0}: Error finding container 9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99: Status 404 returned error can't find the container with id 9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99
Sep 15 21:14:01.635032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:01.635002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt" event={"ID":"56d3349f-0e77-4a1c-b6ef-fb6ef171589a","Type":"ContainerStarted","Data":"9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99"}
Sep 15 21:14:02.642860 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:02.642822    2577 generic.go:356] "Generic (PLEG): container finished" podID="56d3349f-0e77-4a1c-b6ef-fb6ef171589a" containerID="217ece53f17c8855d0b8c90c973ea19ca5b14ef9980e66be89f39106acc02659" exitCode=0
Sep 15 21:14:02.643248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:02.642871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt" event={"ID":"56d3349f-0e77-4a1c-b6ef-fb6ef171589a","Type":"ContainerDied","Data":"217ece53f17c8855d0b8c90c973ea19ca5b14ef9980e66be89f39106acc02659"}
Sep 15 21:14:03.803556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:03.803531    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:03.974882 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:03.974797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca\") pod \"56d3349f-0e77-4a1c-b6ef-fb6ef171589a\" (UID: \"56d3349f-0e77-4a1c-b6ef-fb6ef171589a\") "
Sep 15 21:14:03.975113 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:03.975089    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56d3349f-0e77-4a1c-b6ef-fb6ef171589a" (UID: "56d3349f-0e77-4a1c-b6ef-fb6ef171589a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:14:04.075864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:04.075833    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d3349f-0e77-4a1c-b6ef-fb6ef171589a-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:04.659585 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:04.659531    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"
Sep 15 21:14:04.659758 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:04.659599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt" event={"ID":"56d3349f-0e77-4a1c-b6ef-fb6ef171589a","Type":"ContainerDied","Data":"9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99"}
Sep 15 21:14:04.659758 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:04.659681    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f3432af8fe8a3e71b7e83233d3d55a4795894ab88f733750768099ce9e54a99"
Sep 15 21:14:11.436003 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.435963    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:14:11.436480 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.436224    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" podUID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" containerName="payload-processing" containerID="cri-o://7150f1279ec97c3812ebdbc543b28bc41bee6728b98806a39d048656c5660263" gracePeriod=30
Sep 15 21:14:11.444673 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.444643    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:14:11.444904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.444880    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" podUID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" containerName="payload-pre-processing" containerID="cri-o://64302bfe973bf10fd62c9656de28143fc6f2531f5d187241e82a1975787ba1e7" gracePeriod=30
Sep 15 21:14:11.670969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.670929    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"]
Sep 15 21:14:11.674150 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.674123    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-05731f-r2kmt"]
Sep 15 21:14:11.712206 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.712175    2577 generic.go:356] "Generic (PLEG): container finished" podID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" containerID="7150f1279ec97c3812ebdbc543b28bc41bee6728b98806a39d048656c5660263" exitCode=0
Sep 15 21:14:11.712330 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.712251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" event={"ID":"408e6840-1e4d-4959-9e5f-1e1c24a60ace","Type":"ContainerDied","Data":"7150f1279ec97c3812ebdbc543b28bc41bee6728b98806a39d048656c5660263"}
Sep 15 21:14:11.714397 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.714374    2577 generic.go:356] "Generic (PLEG): container finished" podID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" containerID="64302bfe973bf10fd62c9656de28143fc6f2531f5d187241e82a1975787ba1e7" exitCode=0
Sep 15 21:14:11.714493 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.714440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" event={"ID":"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4","Type":"ContainerDied","Data":"64302bfe973bf10fd62c9656de28143fc6f2531f5d187241e82a1975787ba1e7"}
Sep 15 21:14:11.736584 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.736536    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:14:11.739985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.739964    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:14:11.852119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume\") pod \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") "
Sep 15 21:14:11.852119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nsqs\" (UniqueName: \"kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs\") pod \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") "
Sep 15 21:14:11.852361 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852225    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fp9b\" (UniqueName: \"kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b\") pod \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\" (UID: \"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4\") "
Sep 15 21:14:11.852361 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852298    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume\") pod \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\" (UID: \"408e6840-1e4d-4959-9e5f-1e1c24a60ace\") "
Sep 15 21:14:11.852652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" (UID: "a99dde3a-cd36-4f15-9896-a3ec9a0aaed4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:14:11.852787 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852651    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "408e6840-1e4d-4959-9e5f-1e1c24a60ace" (UID: "408e6840-1e4d-4959-9e5f-1e1c24a60ace"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:14:11.852787 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852761    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/408e6840-1e4d-4959-9e5f-1e1c24a60ace-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:11.852787 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.852782    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:11.854508 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.854478    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b" (OuterVolumeSpecName: "kube-api-access-5fp9b") pod "a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" (UID: "a99dde3a-cd36-4f15-9896-a3ec9a0aaed4"). InnerVolumeSpecName "kube-api-access-5fp9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:14:11.854861 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.854842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs" (OuterVolumeSpecName: "kube-api-access-6nsqs") pod "408e6840-1e4d-4959-9e5f-1e1c24a60ace" (UID: "408e6840-1e4d-4959-9e5f-1e1c24a60ace"). InnerVolumeSpecName "kube-api-access-6nsqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:14:11.953958 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.953932    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fp9b\" (UniqueName: \"kubernetes.io/projected/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4-kube-api-access-5fp9b\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:11.953958 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:11.953956    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nsqs\" (UniqueName: \"kubernetes.io/projected/408e6840-1e4d-4959-9e5f-1e1c24a60ace-kube-api-access-6nsqs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:12.695293 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.695260    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56d3349f-0e77-4a1c-b6ef-fb6ef171589a" path="/var/lib/kubelet/pods/56d3349f-0e77-4a1c-b6ef-fb6ef171589a/volumes"
Sep 15 21:14:12.730017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.729507    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"
Sep 15 21:14:12.730017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.729532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl" event={"ID":"a99dde3a-cd36-4f15-9896-a3ec9a0aaed4","Type":"ContainerDied","Data":"241229ee4bcdd44242f643b815651b8036e7d0c44d70d776e16ba4d514658694"}
Sep 15 21:14:12.730017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.729654    2577 scope.go:117] "RemoveContainer" containerID="64302bfe973bf10fd62c9656de28143fc6f2531f5d187241e82a1975787ba1e7"
Sep 15 21:14:12.732271 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.732212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g" event={"ID":"408e6840-1e4d-4959-9e5f-1e1c24a60ace","Type":"ContainerDied","Data":"6a53cf21fb45485c0316547b3116ee306fbe9b930fff037973cf5c62cbea0b80"}
Sep 15 21:14:12.732437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.732405    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"
Sep 15 21:14:12.741025 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.741006    2577 scope.go:117] "RemoveContainer" containerID="7150f1279ec97c3812ebdbc543b28bc41bee6728b98806a39d048656c5660263"
Sep 15 21:14:12.749932 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.749914    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:14:12.754954 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.754931    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-05731f-df65f86db-qkszl"]
Sep 15 21:14:12.765133 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.765112    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:14:12.768514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:12.768493    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-05731f-6b676db9b5-9745g"]
Sep 15 21:14:14.690636 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:14.690603    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" path="/var/lib/kubelet/pods/408e6840-1e4d-4959-9e5f-1e1c24a60ace/volumes"
Sep 15 21:14:14.691041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:14.690983    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" path="/var/lib/kubelet/pods/a99dde3a-cd36-4f15-9896-a3ec9a0aaed4/volumes"
Sep 15 21:14:24.891507 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.891471    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:14:24.892045 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892008    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" containerName="payload-pre-processing"
Sep 15 21:14:24.892045 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892023    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" containerName="payload-pre-processing"
Sep 15 21:14:24.892045 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56d3349f-0e77-4a1c-b6ef-fb6ef171589a" containerName="revoke-keys"
Sep 15 21:14:24.892045 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892047    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d3349f-0e77-4a1c-b6ef-fb6ef171589a" containerName="revoke-keys"
Sep 15 21:14:24.892276 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892058    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" containerName="payload-processing"
Sep 15 21:14:24.892276 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892067    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" containerName="payload-processing"
Sep 15 21:14:24.892276 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892138    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="408e6840-1e4d-4959-9e5f-1e1c24a60ace" containerName="payload-processing"
Sep 15 21:14:24.892276 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892150    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99dde3a-cd36-4f15-9896-a3ec9a0aaed4" containerName="payload-pre-processing"
Sep 15 21:14:24.892276 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.892157    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="56d3349f-0e77-4a1c-b6ef-fb6ef171589a" containerName="revoke-keys"
Sep 15 21:14:24.897033 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.897014    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.900307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.900286    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"default-dockercfg-bxfmc\""
Sep 15 21:14:24.901327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.901305    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"openshift-service-ca.crt\""
Sep 15 21:14:24.901472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.901311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"kube-root-ca.crt\""
Sep 15 21:14:24.901472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.901315    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"premium-w0-544890-kserve-self-signed-certs\""
Sep 15 21:14:24.908487 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.908118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:14:24.977921 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.977887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.978076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.977959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.978076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.978024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.978076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.978067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.978222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.978091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t8mf\" (UniqueName: \"kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:24.978222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:24.978129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078639 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t8mf\" (UniqueName: \"kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.078963 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.078825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.079037 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.079014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.079183 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.079158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.079183 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.079172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.081227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.081208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.081473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.081457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.088038 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.088017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t8mf\" (UniqueName: \"kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf\") pod \"premium-w0-544890-kserve-78ff654988-sh8fm\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") " pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.211299 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.211217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:25.487610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.487582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:14:25.489198 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:14:25.489174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaedcfa9_b116_44bd_9dc6_8baf87f33d13.slice/crio-31658fb926e4c367eff74a802558d1885c7328acf5d5ad32ef1b8f0ab37ce117 WatchSource:0}: Error finding container 31658fb926e4c367eff74a802558d1885c7328acf5d5ad32ef1b8f0ab37ce117: Status 404 returned error can't find the container with id 31658fb926e4c367eff74a802558d1885c7328acf5d5ad32ef1b8f0ab37ce117
Sep 15 21:14:25.808520 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.808424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" event={"ID":"daedcfa9-b116-44bd-9dc6-8baf87f33d13","Type":"ContainerStarted","Data":"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"}
Sep 15 21:14:25.808520 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.808464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" event={"ID":"daedcfa9-b116-44bd-9dc6-8baf87f33d13","Type":"ContainerStarted","Data":"31658fb926e4c367eff74a802558d1885c7328acf5d5ad32ef1b8f0ab37ce117"}
Sep 15 21:14:25.828812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:25.828756    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" podStartSLOduration=1.828734788 podStartE2EDuration="1.828734788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:14:25.828078808 +0000 UTC m=+973.614446817" watchObservedRunningTime="2026-09-15 21:14:25.828734788 +0000 UTC m=+973.615102798"
Sep 15 21:14:35.211497 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:35.211415    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:35.211497 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:35.211460    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:35.223868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:35.223843    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:35.886221 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:35.886193    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:14:46.262062 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.262023    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:14:46.270917 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.270890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.273916 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.273889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b7100513-dockercfg-5j5v5\""
Sep 15 21:14:46.274166 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.273920    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b7100513\""
Sep 15 21:14:46.275892 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.275873    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:14:46.292936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.292911    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:14:46.297083 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.297066    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.308458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.308437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.308584 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.308489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqctg\" (UniqueName: \"kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.308662 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.308630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.308724 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.308683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8znnw\" (UniqueName: \"kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.313437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.313419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:14:46.409385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.409351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqctg\" (UniqueName: \"kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.409635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.409416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.409635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.409442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znnw\" (UniqueName: \"kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.409635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.409472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.410198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.410178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.410342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.410324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.418031 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.418001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8znnw\" (UniqueName: \"kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw\") pod \"payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") " pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.418150 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.418001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqctg\" (UniqueName: \"kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg\") pod \"payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.583806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.583720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.607207 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.607184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:46.863258 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.863231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:14:46.864858 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:14:46.864832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18bb0b4_694d_4a9c_87d3_3e8d39526ffd.slice/crio-d32f9a8bb59812d0b7fb0ec503e42ac3717a0dbeecf658a4d5436b96628069d5 WatchSource:0}: Error finding container d32f9a8bb59812d0b7fb0ec503e42ac3717a0dbeecf658a4d5436b96628069d5: Status 404 returned error can't find the container with id d32f9a8bb59812d0b7fb0ec503e42ac3717a0dbeecf658a4d5436b96628069d5
Sep 15 21:14:46.891806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.891783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:14:46.893644 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:14:46.893617    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9268426_fc2d_42e2_ac3a_9f24c9cf4844.slice/crio-9c3fe311edcf0e17ae8ca59d5547618e94c63143de06f02ad0ead06a0b060bb1 WatchSource:0}: Error finding container 9c3fe311edcf0e17ae8ca59d5547618e94c63143de06f02ad0ead06a0b060bb1: Status 404 returned error can't find the container with id 9c3fe311edcf0e17ae8ca59d5547618e94c63143de06f02ad0ead06a0b060bb1
Sep 15 21:14:46.941146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.941109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" event={"ID":"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd","Type":"ContainerStarted","Data":"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"}
Sep 15 21:14:46.941304 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.941181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" event={"ID":"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd","Type":"ContainerStarted","Data":"d32f9a8bb59812d0b7fb0ec503e42ac3717a0dbeecf658a4d5436b96628069d5"}
Sep 15 21:14:46.941304 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.941225    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:46.942679 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.942651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" event={"ID":"b9268426-fc2d-42e2-ac3a-9f24c9cf4844","Type":"ContainerStarted","Data":"9c3fe311edcf0e17ae8ca59d5547618e94c63143de06f02ad0ead06a0b060bb1"}
Sep 15 21:14:46.962346 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:46.962302    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" podStartSLOduration=0.962286153 podStartE2EDuration="962.286153ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:14:46.959594805 +0000 UTC m=+994.745962815" watchObservedRunningTime="2026-09-15 21:14:46.962286153 +0000 UTC m=+994.748654161"
Sep 15 21:14:47.950634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:47.950593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" event={"ID":"b9268426-fc2d-42e2-ac3a-9f24c9cf4844","Type":"ContainerStarted","Data":"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"}
Sep 15 21:14:47.951084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:47.950670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:14:47.968922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:47.968873    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" podStartSLOduration=1.968856831 podStartE2EDuration="1.968856831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:14:47.967057823 +0000 UTC m=+995.753425830" watchObservedRunningTime="2026-09-15 21:14:47.968856831 +0000 UTC m=+995.755224839"
Sep 15 21:14:54.013023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.012987    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"]
Sep 15 21:14:54.016912 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.016894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:54.019719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.019693    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:14:54.019833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.019735    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:14:54.026731 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.026707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"]
Sep 15 21:14:54.090167 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.090128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5\" (UID: \"ec812966-bed0-479d-985c-489f4e0e7516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:54.191360 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.191326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5\" (UID: \"ec812966-bed0-479d-985c-489f4e0e7516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:54.191980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.191960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5\" (UID: \"ec812966-bed0-479d-985c-489f4e0e7516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:54.329163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.329070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:54.600803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.600778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"]
Sep 15 21:14:54.602541 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:14:54.602516    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec812966_bed0_479d_985c_489f4e0e7516.slice/crio-7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352 WatchSource:0}: Error finding container 7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352: Status 404 returned error can't find the container with id 7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352
Sep 15 21:14:54.995634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.995595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5" event={"ID":"ec812966-bed0-479d-985c-489f4e0e7516","Type":"ContainerStarted","Data":"1cbe0f8d1f8d132d1f4be07bacad0956d433c5da39c8eed0310f8f8e28e1aaf2"}
Sep 15 21:14:54.995861 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:54.995671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5" event={"ID":"ec812966-bed0-479d-985c-489f4e0e7516","Type":"ContainerStarted","Data":"7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352"}
Sep 15 21:14:55.010755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:55.010707    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5" podStartSLOduration=2.010691253 podStartE2EDuration="2.010691253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:14:55.009364726 +0000 UTC m=+1002.795732737" watchObservedRunningTime="2026-09-15 21:14:55.010691253 +0000 UTC m=+1002.797059261"
Sep 15 21:14:56.002839 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:56.002801    2577 generic.go:356] "Generic (PLEG): container finished" podID="ec812966-bed0-479d-985c-489f4e0e7516" containerID="1cbe0f8d1f8d132d1f4be07bacad0956d433c5da39c8eed0310f8f8e28e1aaf2" exitCode=0
Sep 15 21:14:56.003248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:56.002894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5" event={"ID":"ec812966-bed0-479d-985c-489f4e0e7516","Type":"ContainerDied","Data":"1cbe0f8d1f8d132d1f4be07bacad0956d433c5da39c8eed0310f8f8e28e1aaf2"}
Sep 15 21:14:57.158885 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:57.158855    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:57.222666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:57.222633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca\") pod \"ec812966-bed0-479d-985c-489f4e0e7516\" (UID: \"ec812966-bed0-479d-985c-489f4e0e7516\") "
Sep 15 21:14:57.223033 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:57.223013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ec812966-bed0-479d-985c-489f4e0e7516" (UID: "ec812966-bed0-479d-985c-489f4e0e7516"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:14:57.324255 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:57.324180    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec812966-bed0-479d-985c-489f4e0e7516-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:14:57.953153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:57.953118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:14:58.015683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:58.015656    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"
Sep 15 21:14:58.015843 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:58.015707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5" event={"ID":"ec812966-bed0-479d-985c-489f4e0e7516","Type":"ContainerDied","Data":"7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352"}
Sep 15 21:14:58.015843 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:58.015754    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7840ceefe20ce58036d55299de8476543fbc44381d5482b85f80b15e2148a352"
Sep 15 21:14:58.886331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:58.886300    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:14:58.959690 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:14:58.959665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:15:00.144266 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.144228    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"]
Sep 15 21:15:00.144979 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.144959    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec812966-bed0-479d-985c-489f4e0e7516" containerName="revoke-keys"
Sep 15 21:15:00.145028 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.144985    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec812966-bed0-479d-985c-489f4e0e7516" containerName="revoke-keys"
Sep 15 21:15:00.145151 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.145140    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec812966-bed0-479d-985c-489f4e0e7516" containerName="revoke-keys"
Sep 15 21:15:00.148705 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.148684    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"]
Sep 15 21:15:00.148823 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.148803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"
Sep 15 21:15:00.151649 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.151629    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:15:00.152472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.152444    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"]
Sep 15 21:15:00.152913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.152691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"
Sep 15 21:15:00.156338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.156311    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"]
Sep 15 21:15:00.156500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.156448    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"
Sep 15 21:15:00.159368 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.159344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"]
Sep 15 21:15:00.164445 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.164426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"
Sep 15 21:15:00.175004 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.174982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"
Sep 15 21:15:00.180828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.180807    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"]
Sep 15 21:15:00.218375 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.218349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"
Sep 15 21:15:00.450834 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.450710    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"]
Sep 15 21:15:00.453714 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:00.453684    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1fc651_2ac8_499d_98d5_d867b4963edb.slice/crio-435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c WatchSource:0}: Error finding container 435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c: Status 404 returned error can't find the container with id 435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c
Sep 15 21:15:00.474598 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.474554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"]
Sep 15 21:15:00.477351 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:00.477326    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c292d0f_7b36_4d6b_a8a3_230aaa4d0f2e.slice/crio-247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6 WatchSource:0}: Error finding container 247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6: Status 404 returned error can't find the container with id 247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6
Sep 15 21:15:00.528329 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:00.528306    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"]
Sep 15 21:15:00.530836 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:00.530802    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddccc42ce_33ca_45ed_a64b_cd91057d4927.slice/crio-0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1 WatchSource:0}: Error finding container 0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1: Status 404 returned error can't find the container with id 0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1
Sep 15 21:15:01.037394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.037300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf" event={"ID":"dccc42ce-33ca-45ed-a64b-cd91057d4927","Type":"ContainerStarted","Data":"da313b4d66cf3f3d0b86109d59881fdd034ad421ee4a3998e14d9c38f626d07a"}
Sep 15 21:15:01.037394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.037359    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf" event={"ID":"dccc42ce-33ca-45ed-a64b-cd91057d4927","Type":"ContainerStarted","Data":"0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1"}
Sep 15 21:15:01.038945 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.038919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8" event={"ID":"9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e","Type":"ContainerStarted","Data":"598e91cac1156d0b7cce306cb5f2950e2068fc6db2cb563ce182bf4271d93769"}
Sep 15 21:15:01.039060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.038962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8" event={"ID":"9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e","Type":"ContainerStarted","Data":"247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6"}
Sep 15 21:15:01.040610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.040559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh" event={"ID":"4e1fc651-2ac8-499d-98d5-d867b4963edb","Type":"ContainerStarted","Data":"c38bd30256cd06c65a9aed440db179fb4d06e9ca1950f0f3080f0d4110e389d3"}
Sep 15 21:15:01.040699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.040629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh" event={"ID":"4e1fc651-2ac8-499d-98d5-d867b4963edb","Type":"ContainerStarted","Data":"435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c"}
Sep 15 21:15:01.053547 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.053500    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf" podStartSLOduration=1.053487645 podStartE2EDuration="1.053487645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:01.052911321 +0000 UTC m=+1008.839279330" watchObservedRunningTime="2026-09-15 21:15:01.053487645 +0000 UTC m=+1008.839855653"
Sep 15 21:15:01.067623 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.067559    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8" podStartSLOduration=1.067547086 podStartE2EDuration="1.067547086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:01.066613801 +0000 UTC m=+1008.852981812" watchObservedRunningTime="2026-09-15 21:15:01.067547086 +0000 UTC m=+1008.853915094"
Sep 15 21:15:01.084024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:01.083982    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh" podStartSLOduration=1.0839703410000001 podStartE2EDuration="1.083970341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:01.082300114 +0000 UTC m=+1008.868668124" watchObservedRunningTime="2026-09-15 21:15:01.083970341 +0000 UTC m=+1008.870338345"
Sep 15 21:15:02.048472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.048436    2577 generic.go:356] "Generic (PLEG): container finished" podID="dccc42ce-33ca-45ed-a64b-cd91057d4927" containerID="da313b4d66cf3f3d0b86109d59881fdd034ad421ee4a3998e14d9c38f626d07a" exitCode=0
Sep 15 21:15:02.048924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.048527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf" event={"ID":"dccc42ce-33ca-45ed-a64b-cd91057d4927","Type":"ContainerDied","Data":"da313b4d66cf3f3d0b86109d59881fdd034ad421ee4a3998e14d9c38f626d07a"}
Sep 15 21:15:02.050135 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.050110    2577 generic.go:356] "Generic (PLEG): container finished" podID="9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e" containerID="598e91cac1156d0b7cce306cb5f2950e2068fc6db2cb563ce182bf4271d93769" exitCode=0
Sep 15 21:15:02.050268 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.050215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8" event={"ID":"9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e","Type":"ContainerDied","Data":"598e91cac1156d0b7cce306cb5f2950e2068fc6db2cb563ce182bf4271d93769"}
Sep 15 21:15:02.052054 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.052023    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e1fc651-2ac8-499d-98d5-d867b4963edb" containerID="c38bd30256cd06c65a9aed440db179fb4d06e9ca1950f0f3080f0d4110e389d3" exitCode=0
Sep 15 21:15:02.052151 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.052100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh" event={"ID":"4e1fc651-2ac8-499d-98d5-d867b4963edb","Type":"ContainerDied","Data":"c38bd30256cd06c65a9aed440db179fb4d06e9ca1950f0f3080f0d4110e389d3"}
Sep 15 21:15:02.808933 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.808896    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:15:02.812103 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.812081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.814690 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.814672    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-8f08a9\""
Sep 15 21:15:02.823055 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.823030    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:15:02.838852 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.838820    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:15:02.843421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.843401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:02.848863 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.848844    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-8f08a9-dockercfg-2pmn9\""
Sep 15 21:15:02.854755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.854734    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:15:02.888592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.888523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.888785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.888656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnxpj\" (UniqueName: \"kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.989447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.989416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:02.989676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.989514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnxpj\" (UniqueName: \"kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.989676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.989660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv4tc\" (UniqueName: \"kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:02.989832 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.989761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.990498 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.990478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:02.999320 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:02.999292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnxpj\" (UniqueName: \"kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj\") pod \"payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:03.091819 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.091772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv4tc\" (UniqueName: \"kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:03.092325 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.091899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:03.092796 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.092687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:03.101604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.100272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv4tc\" (UniqueName: \"kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc\") pod \"payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:03.124423 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.124374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:03.156775 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.156278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:03.232031 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.230238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"
Sep 15 21:15:03.278588 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.278532    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"
Sep 15 21:15:03.282485 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.282464    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"
Sep 15 21:15:03.425548 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.425520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:15:03.426897 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:03.426874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b69e5bc_c456_4e43_b3a8_c94653606c7b.slice/crio-62c7756929fd31af079bf8384ebf70d8b316c9de85ba875a7bcc438a1478d1de WatchSource:0}: Error finding container 62c7756929fd31af079bf8384ebf70d8b316c9de85ba875a7bcc438a1478d1de: Status 404 returned error can't find the container with id 62c7756929fd31af079bf8384ebf70d8b316c9de85ba875a7bcc438a1478d1de
Sep 15 21:15:03.452956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:03.451297    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:15:03.464793 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:03.464764    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ed790a_22f9_4e1b_baa9_0a219e480f38.slice/crio-42961e7cb2ff500198f29e14fb1a12f98872763971dfca45b36e741b6641cb7f WatchSource:0}: Error finding container 42961e7cb2ff500198f29e14fb1a12f98872763971dfca45b36e741b6641cb7f: Status 404 returned error can't find the container with id 42961e7cb2ff500198f29e14fb1a12f98872763971dfca45b36e741b6641cb7f
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.067924    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh"
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.067946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825115-r6hvh" event={"ID":"4e1fc651-2ac8-499d-98d5-d867b4963edb","Type":"ContainerDied","Data":"435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c"}
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.068046    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="435edbed5db6b8d503b6e75083c07389ad69387458a3d50c7c1b5e91c132668c"
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.070069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" event={"ID":"53ed790a-22f9-4e1b-baa9-0a219e480f38","Type":"ContainerStarted","Data":"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"}
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.070113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" event={"ID":"53ed790a-22f9-4e1b-baa9-0a219e480f38","Type":"ContainerStarted","Data":"42961e7cb2ff500198f29e14fb1a12f98872763971dfca45b36e741b6641cb7f"}
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.071014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.072967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" event={"ID":"9b69e5bc-c456-4e43-b3a8-c94653606c7b","Type":"ContainerStarted","Data":"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"}
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.073005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" event={"ID":"9b69e5bc-c456-4e43-b3a8-c94653606c7b","Type":"ContainerStarted","Data":"62c7756929fd31af079bf8384ebf70d8b316c9de85ba875a7bcc438a1478d1de"}
Sep 15 21:15:04.074559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.073498    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:04.075828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.075415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf" event={"ID":"dccc42ce-33ca-45ed-a64b-cd91057d4927","Type":"ContainerDied","Data":"0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1"}
Sep 15 21:15:04.075828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.075527    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ff2ab278c43ce81e616d8bca6613de6f3204520ed6c3c8cb9eec9dda21ed2c1"
Sep 15 21:15:04.075828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.075618    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"
Sep 15 21:15:04.078106 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.078086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8" event={"ID":"9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e","Type":"ContainerDied","Data":"247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6"}
Sep 15 21:15:04.078205 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.078125    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="247c76bda76489018c073f9c005cb7b08819c8d565654cafa6d1908eb9290fe6"
Sep 15 21:15:04.078205 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.078177    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"
Sep 15 21:15:04.110637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.110554    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" podStartSLOduration=2.110534898 podStartE2EDuration="2.110534898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:04.088377645 +0000 UTC m=+1011.874745654" watchObservedRunningTime="2026-09-15 21:15:04.110534898 +0000 UTC m=+1011.896902908"
Sep 15 21:15:04.111918 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:04.111866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" podStartSLOduration=2.111854348 podStartE2EDuration="2.111854348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:04.110688879 +0000 UTC m=+1011.897056876" watchObservedRunningTime="2026-09-15 21:15:04.111854348 +0000 UTC m=+1011.898222358"
Sep 15 21:15:05.637890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:05.637849    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:15:05.638768 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:05.638738    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" podUID="f50ad04e-e377-486b-b310-e73fde8df4bb" containerName="payload-processing" containerID="cri-o://00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db" gracePeriod=30
Sep 15 21:15:05.819327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:05.819214    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"]
Sep 15 21:15:05.821440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:05.821394    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-dca193-dszh5"]
Sep 15 21:15:05.918544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:05.918189    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:15:06.022953 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.022926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4wjq\" (UniqueName: \"kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq\") pod \"f50ad04e-e377-486b-b310-e73fde8df4bb\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") "
Sep 15 21:15:06.023109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.023030    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume\") pod \"f50ad04e-e377-486b-b310-e73fde8df4bb\" (UID: \"f50ad04e-e377-486b-b310-e73fde8df4bb\") "
Sep 15 21:15:06.023440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.023417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f50ad04e-e377-486b-b310-e73fde8df4bb" (UID: "f50ad04e-e377-486b-b310-e73fde8df4bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:06.025247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.025211    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq" (OuterVolumeSpecName: "kube-api-access-k4wjq") pod "f50ad04e-e377-486b-b310-e73fde8df4bb" (UID: "f50ad04e-e377-486b-b310-e73fde8df4bb"). InnerVolumeSpecName "kube-api-access-k4wjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:06.092304 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.092271    2577 generic.go:356] "Generic (PLEG): container finished" podID="f50ad04e-e377-486b-b310-e73fde8df4bb" containerID="00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db" exitCode=0
Sep 15 21:15:06.092484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.092327    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"
Sep 15 21:15:06.092484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.092355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" event={"ID":"f50ad04e-e377-486b-b310-e73fde8df4bb","Type":"ContainerDied","Data":"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"}
Sep 15 21:15:06.092484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.092410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr" event={"ID":"f50ad04e-e377-486b-b310-e73fde8df4bb","Type":"ContainerDied","Data":"6f5c2b94a5fd2bdef2e90555d07b653e746e3d01d1f2ffdaddc7f57bc5e96a31"}
Sep 15 21:15:06.092484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.092445    2577 scope.go:117] "RemoveContainer" containerID="00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"
Sep 15 21:15:06.102129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.102113    2577 scope.go:117] "RemoveContainer" containerID="00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"
Sep 15 21:15:06.102428 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:15:06.102399    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db\": container with ID starting with 00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db not found: ID does not exist" containerID="00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"
Sep 15 21:15:06.102549 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.102440    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db"} err="failed to get container status \"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db\": rpc error: code = NotFound desc = could not find container \"00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db\": container with ID starting with 00ec00f6cbeeb7fec498b0b6e440b7893c5e1650147563b70df222fc5a70e7db not found: ID does not exist"
Sep 15 21:15:06.119140 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.119119    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:15:06.124348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.124216    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4wjq\" (UniqueName: \"kubernetes.io/projected/f50ad04e-e377-486b-b310-e73fde8df4bb-kube-api-access-k4wjq\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.124348 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.124244    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50ad04e-e377-486b-b310-e73fde8df4bb-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.126917 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.126898    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-dca193-595779cffc-hqqwr"]
Sep 15 21:15:06.699232 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.699202    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec812966-bed0-479d-985c-489f4e0e7516" path="/var/lib/kubelet/pods/ec812966-bed0-479d-985c-489f4e0e7516/volumes"
Sep 15 21:15:06.699716 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:06.699578    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50ad04e-e377-486b-b310-e73fde8df4bb" path="/var/lib/kubelet/pods/f50ad04e-e377-486b-b310-e73fde8df4bb/volumes"
Sep 15 21:15:09.563557 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.563514    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"]
Sep 15 21:15:09.564379 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564352    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1fc651-2ac8-499d-98d5-d867b4963edb" containerName="cleanup"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564383    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1fc651-2ac8-499d-98d5-d867b4963edb" containerName="cleanup"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564397    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50ad04e-e377-486b-b310-e73fde8df4bb" containerName="payload-processing"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564406    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50ad04e-e377-486b-b310-e73fde8df4bb" containerName="payload-processing"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564433    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e" containerName="cleanup"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564441    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e" containerName="cleanup"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dccc42ce-33ca-45ed-a64b-cd91057d4927" containerName="cleanup"
Sep 15 21:15:09.564514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564475    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dccc42ce-33ca-45ed-a64b-cd91057d4927" containerName="cleanup"
Sep 15 21:15:09.564986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564619    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1fc651-2ac8-499d-98d5-d867b4963edb" containerName="cleanup"
Sep 15 21:15:09.564986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564634    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50ad04e-e377-486b-b310-e73fde8df4bb" containerName="payload-processing"
Sep 15 21:15:09.564986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564646    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dccc42ce-33ca-45ed-a64b-cd91057d4927" containerName="cleanup"
Sep 15 21:15:09.564986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.564656    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e" containerName="cleanup"
Sep 15 21:15:09.568390 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.568364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:09.574592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.574534    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"]
Sep 15 21:15:09.574875 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.574838    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:15:09.574875 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.574867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:15:09.663610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.663534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b7100513-zcpvb\" (UID: \"fc240b13-699f-4ef3-8de9-8b824a0fe7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:09.764819 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.764786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b7100513-zcpvb\" (UID: \"fc240b13-699f-4ef3-8de9-8b824a0fe7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:09.765401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.765383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b7100513-zcpvb\" (UID: \"fc240b13-699f-4ef3-8de9-8b824a0fe7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:09.886677 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:09.886599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:10.202600 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:10.202557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"]
Sep 15 21:15:10.204797 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:10.204773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc240b13_699f_4ef3_8de9_8b824a0fe7ca.slice/crio-5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b WatchSource:0}: Error finding container 5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b: Status 404 returned error can't find the container with id 5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b
Sep 15 21:15:11.129940 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:11.129898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb" event={"ID":"fc240b13-699f-4ef3-8de9-8b824a0fe7ca","Type":"ContainerStarted","Data":"e0a89d4d9d14bfaf33c5481cdb1b26cde843fb7e9fff6196f8095f8c29b82ea8"}
Sep 15 21:15:11.130426 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:11.129965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb" event={"ID":"fc240b13-699f-4ef3-8de9-8b824a0fe7ca","Type":"ContainerStarted","Data":"5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b"}
Sep 15 21:15:11.148929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:11.148887    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb" podStartSLOduration=2.148872214 podStartE2EDuration="2.148872214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:11.14634156 +0000 UTC m=+1018.932709568" watchObservedRunningTime="2026-09-15 21:15:11.148872214 +0000 UTC m=+1018.935240223"
Sep 15 21:15:12.139896 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:12.139861    2577 generic.go:356] "Generic (PLEG): container finished" podID="fc240b13-699f-4ef3-8de9-8b824a0fe7ca" containerID="e0a89d4d9d14bfaf33c5481cdb1b26cde843fb7e9fff6196f8095f8c29b82ea8" exitCode=0
Sep 15 21:15:12.140293 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:12.139943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb" event={"ID":"fc240b13-699f-4ef3-8de9-8b824a0fe7ca","Type":"ContainerDied","Data":"e0a89d4d9d14bfaf33c5481cdb1b26cde843fb7e9fff6196f8095f8c29b82ea8"}
Sep 15 21:15:12.756818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:12.756656    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:15:12.756980 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:12.756941    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" containerID="cri-o://a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268" gracePeriod=30
Sep 15 21:15:13.281177 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:13.281143    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:13.402914 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:13.402837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca\") pod \"fc240b13-699f-4ef3-8de9-8b824a0fe7ca\" (UID: \"fc240b13-699f-4ef3-8de9-8b824a0fe7ca\") "
Sep 15 21:15:13.403220 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:13.403191    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc240b13-699f-4ef3-8de9-8b824a0fe7ca" (UID: "fc240b13-699f-4ef3-8de9-8b824a0fe7ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:13.403308 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:13.403262    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc240b13-699f-4ef3-8de9-8b824a0fe7ca-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:13.906689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:13.906648    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" podUID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" containerName="main" containerID="cri-o://0c42445a0abf2b95669837725e7a959c0d215363d091cda843bea3c4cb80c495" gracePeriod=45
Sep 15 21:15:14.153084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.153061    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"
Sep 15 21:15:14.153233 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.153060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb" event={"ID":"fc240b13-699f-4ef3-8de9-8b824a0fe7ca","Type":"ContainerDied","Data":"5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b"}
Sep 15 21:15:14.153283 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.153256    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5afb01aa58572b0714d696e961e1466873a487400ea28408c9df5a1d5bbba71b"
Sep 15 21:15:14.154681 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.154654    2577 generic.go:356] "Generic (PLEG): container finished" podID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" containerID="0c42445a0abf2b95669837725e7a959c0d215363d091cda843bea3c4cb80c495" exitCode=0
Sep 15 21:15:14.154792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.154730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" event={"ID":"c4bca5ac-ab7a-49af-b308-e95fed00a51c","Type":"ContainerDied","Data":"0c42445a0abf2b95669837725e7a959c0d215363d091cda843bea3c4cb80c495"}
Sep 15 21:15:14.154792 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.154775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz" event={"ID":"c4bca5ac-ab7a-49af-b308-e95fed00a51c","Type":"ContainerDied","Data":"ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3"}
Sep 15 21:15:14.154865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.154790    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca53beab515300b9d141fb663d2c5485b34148169e7e2be34928707c79080db3"
Sep 15 21:15:14.164733 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.164715    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:15:14.312851 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.312814    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.312865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.312902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.312964    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313130    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home" (OuterVolumeSpecName: "home") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313209    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:14.313298 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313278    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hccqs\" (UniqueName: \"kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs\") pod \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\" (UID: \"c4bca5ac-ab7a-49af-b308-e95fed00a51c\") "
Sep 15 21:15:14.313644 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:14.313741 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313725    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tmp-dir\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:14.313789 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313744    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-home\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:14.313789 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.313752    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-model-cache\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:14.315284 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.315263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm" (OuterVolumeSpecName: "dshm") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:14.315560 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.315543    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs" (OuterVolumeSpecName: "kube-api-access-hccqs") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "kube-api-access-hccqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:14.315678 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.315663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4bca5ac-ab7a-49af-b308-e95fed00a51c" (UID: "c4bca5ac-ab7a-49af-b308-e95fed00a51c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:15:14.415376 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.415276    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4bca5ac-ab7a-49af-b308-e95fed00a51c-dshm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:14.415376 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.415316    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4bca5ac-ab7a-49af-b308-e95fed00a51c-tls-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:14.415376 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:14.415331    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hccqs\" (UniqueName: \"kubernetes.io/projected/c4bca5ac-ab7a-49af-b308-e95fed00a51c-kube-api-access-hccqs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:15.160515 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:15.160487    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"
Sep 15 21:15:15.180394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:15.180360    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:15:15.184345 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:15.184318    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-dca193/premium-w4-dca193-kserve-56f9974745-j4sbz"]
Sep 15 21:15:16.094624 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:16.094591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:15:16.095028 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:16.094780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:15:16.588611 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:16.588548    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 21:15:16.588820 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:16.588642    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 21:15:16.690584 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:16.690542    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" path="/var/lib/kubelet/pods/c4bca5ac-ab7a-49af-b308-e95fed00a51c/volumes"
Sep 15 21:15:18.011264 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.011238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:15:18.154531 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154531 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154493    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154733 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154535    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154733 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154709    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154746    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154781    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:18.154890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9rqt\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154851    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.154890 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154881    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.155182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154949    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket\") pod \"59c7a38b-af66-4a5e-9180-2451858939da\" (UID: \"59c7a38b-af66-4a5e-9180-2451858939da\") "
Sep 15 21:15:18.155182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.154979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:18.155182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data" (OuterVolumeSpecName: "istio-data") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:18.155347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:18.155347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155283    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:18.155652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155526    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.155652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155546    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59c7a38b-af66-4a5e-9180-2451858939da-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.155652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155560    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.155652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155592    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.155652 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.155603    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.157065 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.157039    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token" (OuterVolumeSpecName: "istio-token") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:18.157186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.157040    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt" (OuterVolumeSpecName: "kube-api-access-x9rqt") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "kube-api-access-x9rqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:18.157186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.157080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:15:18.157351 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.157327    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59c7a38b-af66-4a5e-9180-2451858939da" (UID: "59c7a38b-af66-4a5e-9180-2451858939da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:15:18.181706 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.181679    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c7a38b-af66-4a5e-9180-2451858939da" containerID="a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268" exitCode=0
Sep 15 21:15:18.181835 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.181729    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"
Sep 15 21:15:18.181835 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.181761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerDied","Data":"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"}
Sep 15 21:15:18.181835 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.181816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r" event={"ID":"59c7a38b-af66-4a5e-9180-2451858939da","Type":"ContainerDied","Data":"3178b80181fa4e6ff7102e2eeab393637c40e0ebadd5394412d0e52ab36ce668"}
Sep 15 21:15:18.181938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.181839    2577 scope.go:117] "RemoveContainer" containerID="a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"
Sep 15 21:15:18.191518 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.191499    2577 scope.go:117] "RemoveContainer" containerID="5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"
Sep 15 21:15:18.200146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.200123    2577 scope.go:117] "RemoveContainer" containerID="a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"
Sep 15 21:15:18.200392 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:15:18.200373    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268\": container with ID starting with a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268 not found: ID does not exist" containerID="a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"
Sep 15 21:15:18.200442 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.200401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268"} err="failed to get container status \"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268\": rpc error: code = NotFound desc = could not find container \"a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268\": container with ID starting with a381fcab9d90bcfd6642457ddbf6e2dd9d34d5489f07696900527c99bc432268 not found: ID does not exist"
Sep 15 21:15:18.200442 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.200417    2577 scope.go:117] "RemoveContainer" containerID="5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"
Sep 15 21:15:18.200704 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:15:18.200689    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c\": container with ID starting with 5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c not found: ID does not exist" containerID="5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"
Sep 15 21:15:18.200756 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.200709    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c"} err="failed to get container status \"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c\": rpc error: code = NotFound desc = could not find container \"5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c\": container with ID starting with 5dbf8b899e10b6deb3a348dc67b0b79a569da867f4877d03d9d0ae407761b40c not found: ID does not exist"
Sep 15 21:15:18.205391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.205368    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:15:18.209121 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.209099    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-dca193-openshift-default-7b9785c5cd-tbk2r"]
Sep 15 21:15:18.257119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.257092    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.257119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.257113    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59c7a38b-af66-4a5e-9180-2451858939da-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.257119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.257122    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59c7a38b-af66-4a5e-9180-2451858939da-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.257306 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.257132    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9rqt\" (UniqueName: \"kubernetes.io/projected/59c7a38b-af66-4a5e-9180-2451858939da-kube-api-access-x9rqt\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:18.691322 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:18.691290    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c7a38b-af66-4a5e-9180-2451858939da" path="/var/lib/kubelet/pods/59c7a38b-af66-4a5e-9180-2451858939da/volumes"
Sep 15 21:15:23.738199 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.738163    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"]
Sep 15 21:15:23.740479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.740451    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b7100513-29825115-m58r8"]
Sep 15 21:15:23.767882 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.767849    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:15:23.768163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.768125    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" podUID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" containerName="payload-processing" containerID="cri-o://a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505" gracePeriod=30
Sep 15 21:15:23.772191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.772166    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:15:23.772600 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.772527    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" podUID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" containerName="payload-pre-processing" containerID="cri-o://cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab" gracePeriod=30
Sep 15 21:15:23.931710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.931612    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"]
Sep 15 21:15:23.935500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:23.935460    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b7100513-zcpvb"]
Sep 15 21:15:24.083451 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.083424    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:15:24.086741 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.086722    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:15:24.116419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116388    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume\") pod \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") "
Sep 15 21:15:24.116609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume\") pod \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") "
Sep 15 21:15:24.116609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqctg\" (UniqueName: \"kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg\") pod \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\" (UID: \"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd\") "
Sep 15 21:15:24.116609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8znnw\" (UniqueName: \"kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw\") pod \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\" (UID: \"b9268426-fc2d-42e2-ac3a-9f24c9cf4844\") "
Sep 15 21:15:24.116864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116838    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" (UID: "b18bb0b4-694d-4a9c-87d3-3e8d39526ffd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:24.116926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.116877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9268426-fc2d-42e2-ac3a-9f24c9cf4844" (UID: "b9268426-fc2d-42e2-ac3a-9f24c9cf4844"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:24.119519 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.119153    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw" (OuterVolumeSpecName: "kube-api-access-8znnw") pod "b9268426-fc2d-42e2-ac3a-9f24c9cf4844" (UID: "b9268426-fc2d-42e2-ac3a-9f24c9cf4844"). InnerVolumeSpecName "kube-api-access-8znnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:24.119519 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.119431    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg" (OuterVolumeSpecName: "kube-api-access-tqctg") pod "b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" (UID: "b18bb0b4-694d-4a9c-87d3-3e8d39526ffd"). InnerVolumeSpecName "kube-api-access-tqctg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:24.217391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.217359    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:24.217391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.217390    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqctg\" (UniqueName: \"kubernetes.io/projected/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd-kube-api-access-tqctg\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:24.217627 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.217401    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8znnw\" (UniqueName: \"kubernetes.io/projected/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-kube-api-access-8znnw\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:24.217627 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.217410    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9268426-fc2d-42e2-ac3a-9f24c9cf4844-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:15:24.223805 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.223777    2577 generic.go:356] "Generic (PLEG): container finished" podID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" containerID="cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab" exitCode=0
Sep 15 21:15:24.223922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.223822    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"
Sep 15 21:15:24.223922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.223857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" event={"ID":"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd","Type":"ContainerDied","Data":"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"}
Sep 15 21:15:24.223922 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.223907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c" event={"ID":"b18bb0b4-694d-4a9c-87d3-3e8d39526ffd","Type":"ContainerDied","Data":"d32f9a8bb59812d0b7fb0ec503e42ac3717a0dbeecf658a4d5436b96628069d5"}
Sep 15 21:15:24.224072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.223930    2577 scope.go:117] "RemoveContainer" containerID="cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"
Sep 15 21:15:24.225306 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.225281    2577 generic.go:356] "Generic (PLEG): container finished" podID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" containerID="a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505" exitCode=0
Sep 15 21:15:24.225440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.225317    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"
Sep 15 21:15:24.225440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.225363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" event={"ID":"b9268426-fc2d-42e2-ac3a-9f24c9cf4844","Type":"ContainerDied","Data":"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"}
Sep 15 21:15:24.225440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.225412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz" event={"ID":"b9268426-fc2d-42e2-ac3a-9f24c9cf4844","Type":"ContainerDied","Data":"9c3fe311edcf0e17ae8ca59d5547618e94c63143de06f02ad0ead06a0b060bb1"}
Sep 15 21:15:24.235579 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.235547    2577 scope.go:117] "RemoveContainer" containerID="cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"
Sep 15 21:15:24.236090 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:15:24.236051    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab\": container with ID starting with cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab not found: ID does not exist" containerID="cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"
Sep 15 21:15:24.236172 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.236089    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab"} err="failed to get container status \"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab\": rpc error: code = NotFound desc = could not find container \"cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab\": container with ID starting with cf4a32ad10f27264fdb1f634beefb2bf2b436b2ac42cf2ecf7c340e403ab5bab not found: ID does not exist"
Sep 15 21:15:24.236172 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.236115    2577 scope.go:117] "RemoveContainer" containerID="a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"
Sep 15 21:15:24.246022 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.246004    2577 scope.go:117] "RemoveContainer" containerID="a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"
Sep 15 21:15:24.246278 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:15:24.246263    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505\": container with ID starting with a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505 not found: ID does not exist" containerID="a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"
Sep 15 21:15:24.246327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.246283    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505"} err="failed to get container status \"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505\": rpc error: code = NotFound desc = could not find container \"a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505\": container with ID starting with a0a1fd3cf667eb89eb720179347eddba93f35510999ebc1ff422ecd08b629505 not found: ID does not exist"
Sep 15 21:15:24.251337 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.251316    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:15:24.254799 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.254771    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b7100513-dd6c5c669-r96nz"]
Sep 15 21:15:24.265552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.265525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:15:24.271087 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.271062    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b7100513-7c867d59fd-khc7c"]
Sep 15 21:15:24.690252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.690214    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e" path="/var/lib/kubelet/pods/9c292d0f-7b36-4d6b-a8a3-230aaa4d0f2e/volumes"
Sep 15 21:15:24.690594 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.690552    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" path="/var/lib/kubelet/pods/b18bb0b4-694d-4a9c-87d3-3e8d39526ffd/volumes"
Sep 15 21:15:24.691094 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.691076    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" path="/var/lib/kubelet/pods/b9268426-fc2d-42e2-ac3a-9f24c9cf4844/volumes"
Sep 15 21:15:24.691414 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:24.691402    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc240b13-699f-4ef3-8de9-8b824a0fe7ca" path="/var/lib/kubelet/pods/fc240b13-699f-4ef3-8de9-8b824a0fe7ca/volumes"
Sep 15 21:15:53.201141 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201106    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201612    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201627    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201641    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc240b13-699f-4ef3-8de9-8b824a0fe7ca" containerName="revoke-keys"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201646    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc240b13-699f-4ef3-8de9-8b824a0fe7ca" containerName="revoke-keys"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" containerName="payload-pre-processing"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201662    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" containerName="payload-pre-processing"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201671    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" containerName="main"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201677    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" containerName="main"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201684    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201689    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201694    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" containerName="payload-processing"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201699    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" containerName="payload-processing"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201767    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9268426-fc2d-42e2-ac3a-9f24c9cf4844" containerName="payload-processing"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201775    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4bca5ac-ab7a-49af-b308-e95fed00a51c" containerName="main"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201782    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc240b13-699f-4ef3-8de9-8b824a0fe7ca" containerName="revoke-keys"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201788    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201794    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c7a38b-af66-4a5e-9180-2451858939da" containerName="istio-proxy"
Sep 15 21:15:53.203421 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.201798    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18bb0b4-694d-4a9c-87d3-3e8d39526ffd" containerName="payload-pre-processing"
Sep 15 21:15:53.204738 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.204723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.207546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.207524    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-7c59a9-dockercfg-tczn4\""
Sep 15 21:15:53.208073 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.208056    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-7c59a9\""
Sep 15 21:15:53.219105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.219084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:15:53.264267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.264235    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:15:53.267824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.267808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.279872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.279773    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:15:53.309105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.309066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.309422 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.309393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpkp8\" (UniqueName: \"kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.410971 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.410928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x676k\" (UniqueName: \"kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.410971 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.410981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.411255 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.411094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpkp8\" (UniqueName: \"kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.411255 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.411175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.411744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.411721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.420010 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.419990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpkp8\" (UniqueName: \"kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8\") pod \"payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.512011 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.511973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.512211 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.512066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x676k\" (UniqueName: \"kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.512685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.512666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.515808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.515794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:53.521246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.521214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x676k\" (UniqueName: \"kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k\") pod \"payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.581884 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.580397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:53.788976 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.788948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:15:53.790181 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:53.790154    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2878d0_2df8_4f72_8954_0612fb06695b.slice/crio-1d0a2b66bb1ab6714e88f8507a582990bffb0d28534f9ea2543b625bd040f34b WatchSource:0}: Error finding container 1d0a2b66bb1ab6714e88f8507a582990bffb0d28534f9ea2543b625bd040f34b: Status 404 returned error can't find the container with id 1d0a2b66bb1ab6714e88f8507a582990bffb0d28534f9ea2543b625bd040f34b
Sep 15 21:15:53.866082 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:53.866060    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:15:53.867744 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:15:53.867721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1a4ef3_7927_4111_9485_9a893d24c6d0.slice/crio-bd092150433e6be8f740ff87b336293e7194941f92d28e816def26e7463b6b7a WatchSource:0}: Error finding container bd092150433e6be8f740ff87b336293e7194941f92d28e816def26e7463b6b7a: Status 404 returned error can't find the container with id bd092150433e6be8f740ff87b336293e7194941f92d28e816def26e7463b6b7a
Sep 15 21:15:54.408598 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.408525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" event={"ID":"1f2878d0-2df8-4f72-8954-0612fb06695b","Type":"ContainerStarted","Data":"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"}
Sep 15 21:15:54.409066 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.408616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" event={"ID":"1f2878d0-2df8-4f72-8954-0612fb06695b","Type":"ContainerStarted","Data":"1d0a2b66bb1ab6714e88f8507a582990bffb0d28534f9ea2543b625bd040f34b"}
Sep 15 21:15:54.409066 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.408743    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:15:54.410294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.410272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" event={"ID":"7a1a4ef3-7927-4111-9485-9a893d24c6d0","Type":"ContainerStarted","Data":"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"}
Sep 15 21:15:54.410419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.410314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" event={"ID":"7a1a4ef3-7927-4111-9485-9a893d24c6d0","Type":"ContainerStarted","Data":"bd092150433e6be8f740ff87b336293e7194941f92d28e816def26e7463b6b7a"}
Sep 15 21:15:54.410419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.410343    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:15:54.431346 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.431293    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" podStartSLOduration=1.431275752 podStartE2EDuration="1.431275752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:54.429919154 +0000 UTC m=+1062.216287163" watchObservedRunningTime="2026-09-15 21:15:54.431275752 +0000 UTC m=+1062.217643761"
Sep 15 21:15:54.451479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:15:54.451424    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" podStartSLOduration=1.451408574 podStartE2EDuration="1.451408574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:54.449735459 +0000 UTC m=+1062.236103466" watchObservedRunningTime="2026-09-15 21:15:54.451408574 +0000 UTC m=+1062.237776620"
Sep 15 21:16:01.595124 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.595090    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"]
Sep 15 21:16:01.599154 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.599137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:01.601747 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.601726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:16:01.601868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.601726    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:16:01.606791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.606765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"]
Sep 15 21:16:01.696506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.696478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx\" (UID: \"e503ad84-9e8e-4732-967d-c11f38b4a9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:01.797783 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.797747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx\" (UID: \"e503ad84-9e8e-4732-967d-c11f38b4a9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:01.798354 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.798329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx\" (UID: \"e503ad84-9e8e-4732-967d-c11f38b4a9c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:01.913151 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:01.913044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:02.189349 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:02.189324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"]
Sep 15 21:16:02.191094 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:16:02.191068    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode503ad84_9e8e_4732_967d_c11f38b4a9c3.slice/crio-7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d WatchSource:0}: Error finding container 7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d: Status 404 returned error can't find the container with id 7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d
Sep 15 21:16:02.466212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:02.466170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx" event={"ID":"e503ad84-9e8e-4732-967d-c11f38b4a9c3","Type":"ContainerStarted","Data":"fbf360d3cde1582e445f8bf3780330e630453c73bd5397624152938a5ece9ae4"}
Sep 15 21:16:02.466383 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:02.466237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx" event={"ID":"e503ad84-9e8e-4732-967d-c11f38b4a9c3","Type":"ContainerStarted","Data":"7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d"}
Sep 15 21:16:02.485719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:02.485672    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx" podStartSLOduration=1.485655572 podStartE2EDuration="1.485655572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:16:02.484084005 +0000 UTC m=+1070.270452014" watchObservedRunningTime="2026-09-15 21:16:02.485655572 +0000 UTC m=+1070.272023606"
Sep 15 21:16:03.473376 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:03.473335    2577 generic.go:356] "Generic (PLEG): container finished" podID="e503ad84-9e8e-4732-967d-c11f38b4a9c3" containerID="fbf360d3cde1582e445f8bf3780330e630453c73bd5397624152938a5ece9ae4" exitCode=0
Sep 15 21:16:03.473812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:03.473426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx" event={"ID":"e503ad84-9e8e-4732-967d-c11f38b4a9c3","Type":"ContainerDied","Data":"fbf360d3cde1582e445f8bf3780330e630453c73bd5397624152938a5ece9ae4"}
Sep 15 21:16:04.640686 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:04.640661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:04.725781 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:04.725757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca\") pod \"e503ad84-9e8e-4732-967d-c11f38b4a9c3\" (UID: \"e503ad84-9e8e-4732-967d-c11f38b4a9c3\") "
Sep 15 21:16:04.726067 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:04.726040    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e503ad84-9e8e-4732-967d-c11f38b4a9c3" (UID: "e503ad84-9e8e-4732-967d-c11f38b4a9c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:16:04.827383 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:04.827303    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e503ad84-9e8e-4732-967d-c11f38b4a9c3-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:16:05.418060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:05.418033    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:16:05.418280 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:05.418213    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:16:05.487616 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:05.487585    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"
Sep 15 21:16:05.487803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:05.487614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx" event={"ID":"e503ad84-9e8e-4732-967d-c11f38b4a9c3","Type":"ContainerDied","Data":"7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d"}
Sep 15 21:16:05.487803 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:05.487685    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c1f6d75740439189430c252bbf5f519c1a0dc4a3dead22ec5b0faa3f35d350d"
Sep 15 21:16:09.110780 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.110656    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:16:09.111367 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.111347    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e503ad84-9e8e-4732-967d-c11f38b4a9c3" containerName="revoke-keys"
Sep 15 21:16:09.111450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.111370    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e503ad84-9e8e-4732-967d-c11f38b4a9c3" containerName="revoke-keys"
Sep 15 21:16:09.111512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.111491    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e503ad84-9e8e-4732-967d-c11f38b4a9c3" containerName="revoke-keys"
Sep 15 21:16:09.115089 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.115068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.118109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.118084    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-2c7774-openshift-default-dockercfg-smlvh\""
Sep 15 21:16:09.133904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.133877    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:16:09.170461 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170615    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.170844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.170780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hc5\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hc5\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272782 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272908 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.272969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.273022 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.272964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.274758 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.274731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.275003 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.274983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.285124 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.285096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hc5\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.286219 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.286186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token\") pod \"e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") " pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.426160 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.426074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:09.709364 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.709329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:16:09.711811 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:16:09.711783    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf514ad_b83d_4498_ad21_c4c093a5212b.slice/crio-2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d WatchSource:0}: Error finding container 2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d: Status 404 returned error can't find the container with id 2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d
Sep 15 21:16:09.714282 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.714252    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:09.714360 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.714322    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:09.714360 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:09.714349    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:10.526713 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:10.526671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerStarted","Data":"b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b"}
Sep 15 21:16:10.527089 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:10.526733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerStarted","Data":"2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d"}
Sep 15 21:16:10.549827 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:10.549767    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podStartSLOduration=1.549746931 podStartE2EDuration="1.549746931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:16:10.548309396 +0000 UTC m=+1078.334677403" watchObservedRunningTime="2026-09-15 21:16:10.549746931 +0000 UTC m=+1078.336114940"
Sep 15 21:16:11.426770 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:11.426737    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:11.432414 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:11.432388    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:11.536635 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:11.536606    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:11.537689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:11.537661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:13.011895 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.011856    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:16:13.012349 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.012072    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" podUID="53ed790a-22f9-4e1b-baa9-0a219e480f38" containerName="payload-processing" containerID="cri-o://63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b" gracePeriod=30
Sep 15 21:16:13.024178 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.024111    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:16:13.024383 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.024343    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" podUID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" containerName="payload-pre-processing" containerID="cri-o://ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494" gracePeriod=30
Sep 15 21:16:13.213619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.213579    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"]
Sep 15 21:16:13.216461 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.216428    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-8f08a9-8crgx"]
Sep 15 21:16:13.286415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.286350    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:16:13.322828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.322793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:16:13.419478 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419450    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume\") pod \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") "
Sep 15 21:16:13.419711 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv4tc\" (UniqueName: \"kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc\") pod \"53ed790a-22f9-4e1b-baa9-0a219e480f38\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") "
Sep 15 21:16:13.419711 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume\") pod \"53ed790a-22f9-4e1b-baa9-0a219e480f38\" (UID: \"53ed790a-22f9-4e1b-baa9-0a219e480f38\") "
Sep 15 21:16:13.419711 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419673    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnxpj\" (UniqueName: \"kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj\") pod \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\" (UID: \"9b69e5bc-c456-4e43-b3a8-c94653606c7b\") "
Sep 15 21:16:13.419913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b69e5bc-c456-4e43-b3a8-c94653606c7b" (UID: "9b69e5bc-c456-4e43-b3a8-c94653606c7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:16:13.419988 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.419968    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53ed790a-22f9-4e1b-baa9-0a219e480f38" (UID: "53ed790a-22f9-4e1b-baa9-0a219e480f38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:16:13.420099 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.420084    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b69e5bc-c456-4e43-b3a8-c94653606c7b-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:16:13.420141 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.420101    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ed790a-22f9-4e1b-baa9-0a219e480f38-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:16:13.421852 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.421824    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj" (OuterVolumeSpecName: "kube-api-access-nnxpj") pod "9b69e5bc-c456-4e43-b3a8-c94653606c7b" (UID: "9b69e5bc-c456-4e43-b3a8-c94653606c7b"). InnerVolumeSpecName "kube-api-access-nnxpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:16:13.421969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.421780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc" (OuterVolumeSpecName: "kube-api-access-vv4tc") pod "53ed790a-22f9-4e1b-baa9-0a219e480f38" (UID: "53ed790a-22f9-4e1b-baa9-0a219e480f38"). InnerVolumeSpecName "kube-api-access-vv4tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:16:13.520737 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.520704    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnxpj\" (UniqueName: \"kubernetes.io/projected/9b69e5bc-c456-4e43-b3a8-c94653606c7b-kube-api-access-nnxpj\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:16:13.520737 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.520732    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv4tc\" (UniqueName: \"kubernetes.io/projected/53ed790a-22f9-4e1b-baa9-0a219e480f38-kube-api-access-vv4tc\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:16:13.550483 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.550395    2577 generic.go:356] "Generic (PLEG): container finished" podID="53ed790a-22f9-4e1b-baa9-0a219e480f38" containerID="63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b" exitCode=0
Sep 15 21:16:13.550483 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.550444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"
Sep 15 21:16:13.550755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.550488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" event={"ID":"53ed790a-22f9-4e1b-baa9-0a219e480f38","Type":"ContainerDied","Data":"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"}
Sep 15 21:16:13.550755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.550543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w" event={"ID":"53ed790a-22f9-4e1b-baa9-0a219e480f38","Type":"ContainerDied","Data":"42961e7cb2ff500198f29e14fb1a12f98872763971dfca45b36e741b6641cb7f"}
Sep 15 21:16:13.550755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.550606    2577 scope.go:117] "RemoveContainer" containerID="63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"
Sep 15 21:16:13.551948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.551899    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" containerID="ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494" exitCode=0
Sep 15 21:16:13.552085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.551953    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"
Sep 15 21:16:13.552085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.551980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" event={"ID":"9b69e5bc-c456-4e43-b3a8-c94653606c7b","Type":"ContainerDied","Data":"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"}
Sep 15 21:16:13.552085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.552030    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5" event={"ID":"9b69e5bc-c456-4e43-b3a8-c94653606c7b","Type":"ContainerDied","Data":"62c7756929fd31af079bf8384ebf70d8b316c9de85ba875a7bcc438a1478d1de"}
Sep 15 21:16:13.561153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.561137    2577 scope.go:117] "RemoveContainer" containerID="63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"
Sep 15 21:16:13.561523 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:16:13.561412    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b\": container with ID starting with 63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b not found: ID does not exist" containerID="63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"
Sep 15 21:16:13.561523 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.561445    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b"} err="failed to get container status \"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b\": rpc error: code = NotFound desc = could not find container \"63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b\": container with ID starting with 63299a162e48f95fb81526ba1f50f0a20e35492d3566b0d29f41002625405f5b not found: ID does not exist"
Sep 15 21:16:13.561523 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.561468    2577 scope.go:117] "RemoveContainer" containerID="ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"
Sep 15 21:16:13.569889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.569873    2577 scope.go:117] "RemoveContainer" containerID="ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"
Sep 15 21:16:13.570157 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:16:13.570139    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494\": container with ID starting with ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494 not found: ID does not exist" containerID="ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"
Sep 15 21:16:13.570216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.570163    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494"} err="failed to get container status \"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494\": rpc error: code = NotFound desc = could not find container \"ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494\": container with ID starting with ab6f78f202688caa079eae1ea0f3e449abc2a1186fd5c054a9d2ffc1e1bd0494 not found: ID does not exist"
Sep 15 21:16:13.576259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.576237    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:16:13.585946 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.585928    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f08a9-85848cc5f7-9k54w"]
Sep 15 21:16:13.597631 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.597609    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:16:13.605537 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:13.605514    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f08a9-5b48d98c7d-6v8l5"]
Sep 15 21:16:14.690797 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:14.690767    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53ed790a-22f9-4e1b-baa9-0a219e480f38" path="/var/lib/kubelet/pods/53ed790a-22f9-4e1b-baa9-0a219e480f38/volumes"
Sep 15 21:16:14.691173 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:14.691158    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" path="/var/lib/kubelet/pods/9b69e5bc-c456-4e43-b3a8-c94653606c7b/volumes"
Sep 15 21:16:14.691445 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:14.691432    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e503ad84-9e8e-4732-967d-c11f38b4a9c3" path="/var/lib/kubelet/pods/e503ad84-9e8e-4732-967d-c11f38b4a9c3/volumes"
Sep 15 21:16:17.254699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.254661    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:16:17.255258 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255221    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" containerName="payload-pre-processing"
Sep 15 21:16:17.255258 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255245    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" containerName="payload-pre-processing"
Sep 15 21:16:17.255475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255284    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53ed790a-22f9-4e1b-baa9-0a219e480f38" containerName="payload-processing"
Sep 15 21:16:17.255475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255293    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="53ed790a-22f9-4e1b-baa9-0a219e480f38" containerName="payload-processing"
Sep 15 21:16:17.255475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255408    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="53ed790a-22f9-4e1b-baa9-0a219e480f38" containerName="payload-processing"
Sep 15 21:16:17.255475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.255423    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b69e5bc-c456-4e43-b3a8-c94653606c7b" containerName="payload-pre-processing"
Sep 15 21:16:17.260762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.260740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.264107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.264080    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-2c7774\""
Sep 15 21:16:17.266431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.266409    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:16:17.271042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.271024    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.271130 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.271023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:16:17.273676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.273655    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-2c7774-dockercfg-b756s\""
Sep 15 21:16:17.281017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.280993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:16:17.360767 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.360730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spc88\" (UniqueName: \"kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.360933 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.360852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.361013 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.360943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.361013 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.360985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76g7l\" (UniqueName: \"kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.462032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.461995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spc88\" (UniqueName: \"kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.462223 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.462048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.462223 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.462088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.462223 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.462116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76g7l\" (UniqueName: \"kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.462860 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.462833    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.462860 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.462848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.474289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.474263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spc88\" (UniqueName: \"kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88\") pod \"payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.474394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.474305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76g7l\" (UniqueName: \"kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l\") pod \"payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.572981 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.572899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:17.584550 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.584526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:17.867462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.867430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:16:17.868120 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:16:17.868092    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe59df2_77f5_49b0_92b0_b12124af31c7.slice/crio-6e29c5060b74ca12b7a862f21d734c25a8290bca2f7c2893f87045b56dafc0bb WatchSource:0}: Error finding container 6e29c5060b74ca12b7a862f21d734c25a8290bca2f7c2893f87045b56dafc0bb: Status 404 returned error can't find the container with id 6e29c5060b74ca12b7a862f21d734c25a8290bca2f7c2893f87045b56dafc0bb
Sep 15 21:16:17.909605 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:17.909579    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:16:17.910102 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:16:17.910078    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84a630a_2886_425f_b279_3cbed2ac3072.slice/crio-5bfb9aa6aa24ce94b529f12070e3e97d27bf499be431dae1e066eb543febee82 WatchSource:0}: Error finding container 5bfb9aa6aa24ce94b529f12070e3e97d27bf499be431dae1e066eb543febee82: Status 404 returned error can't find the container with id 5bfb9aa6aa24ce94b529f12070e3e97d27bf499be431dae1e066eb543febee82
Sep 15 21:16:18.586730 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.586693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" event={"ID":"c84a630a-2886-425f-b279-3cbed2ac3072","Type":"ContainerStarted","Data":"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"}
Sep 15 21:16:18.587184 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.586751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" event={"ID":"c84a630a-2886-425f-b279-3cbed2ac3072","Type":"ContainerStarted","Data":"5bfb9aa6aa24ce94b529f12070e3e97d27bf499be431dae1e066eb543febee82"}
Sep 15 21:16:18.587184 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.586883    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:18.588499 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.588472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" event={"ID":"2fe59df2-77f5-49b0-92b0-b12124af31c7","Type":"ContainerStarted","Data":"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"}
Sep 15 21:16:18.588678 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.588526    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:18.588678 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.588558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" event={"ID":"2fe59df2-77f5-49b0-92b0-b12124af31c7","Type":"ContainerStarted","Data":"6e29c5060b74ca12b7a862f21d734c25a8290bca2f7c2893f87045b56dafc0bb"}
Sep 15 21:16:18.605236 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.605191    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" podStartSLOduration=1.605178296 podStartE2EDuration="1.605178296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:16:18.603421177 +0000 UTC m=+1086.389789187" watchObservedRunningTime="2026-09-15 21:16:18.605178296 +0000 UTC m=+1086.391546304"
Sep 15 21:16:18.628491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:18.628448    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" podStartSLOduration=1.6284346699999999 podStartE2EDuration="1.62843467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:16:18.626863316 +0000 UTC m=+1086.413231325" watchObservedRunningTime="2026-09-15 21:16:18.62843467 +0000 UTC m=+1086.414802678"
Sep 15 21:16:29.596718 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:29.596683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:16:29.597303 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:29.596758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:16:42.303063 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.303023    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": read tcp 10.134.0.2:41680->10.134.0.75:15021: read: connection reset by peer" start-of-body=
Sep 15 21:16:42.303496 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.303096    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": read tcp 10.134.0.2:41680->10.134.0.75:15021: read: connection reset by peer"
Sep 15 21:16:42.742900 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.742871    2577 generic.go:356] "Generic (PLEG): container finished" podID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerID="b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b" exitCode=0
Sep 15 21:16:42.743080 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.742913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerDied","Data":"b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b"}
Sep 15 21:16:42.743317 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.743303    2577 scope.go:117] "RemoveContainer" containerID="b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b"
Sep 15 21:16:42.744641 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.744605    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:42.744749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.744683    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:42.744749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:42.744728    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:16:43.754724 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:43.754681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerStarted","Data":"5f986c96be151db56db6d9e877a7a39d8c4c28c04b9be4e5807c5b973c8e6174"}
Sep 15 21:16:44.427020 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:44.426976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:44.427205 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:44.427144    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 21:16:44.427250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:44.427195    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 21:16:45.426629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:45.426595    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 21:16:45.427017 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:45.426657    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 21:16:46.427148 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:46.427116    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 21:16:46.427586 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:46.427169    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 21:16:47.426661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:47.426629    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 21:16:47.427082 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:47.426693    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 21:16:48.426703 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:48.426667    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 15 21:16:48.427154 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:48.426728    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 15 21:16:49.431085 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:49.431057    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:49.431498 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:49.431416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:16:49.432260 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:16:49.432239    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:17:00.195912 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.195875    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:17:00.199491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.199467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.202698 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.202651    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-56c6b012\""
Sep 15 21:17:00.202836 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.202810    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-56c6b012-dockercfg-qtfbv\""
Sep 15 21:17:00.211186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.211162    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:17:00.227228 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.227198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:17:00.229962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.229940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.242000 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.241977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:17:00.320637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.320601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs24z\" (UniqueName: \"kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.321025 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.320999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.321309 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.321288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td9m6\" (UniqueName: \"kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.321536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.321503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.422951 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.422918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td9m6\" (UniqueName: \"kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.423164 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.422973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.423164 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.423016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs24z\" (UniqueName: \"kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.423164 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.423049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.423763 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.423743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.423818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.423759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.433877 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.433854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs24z\" (UniqueName: \"kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z\") pod \"payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.433998 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.433852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td9m6\" (UniqueName: \"kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6\") pod \"payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") " pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.511732 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.511704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:00.542461 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.542432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:00.789217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.789120    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:17:00.795143 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:17:00.795110    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993b6734_50fa_4b48_9f97_9603a313f472.slice/crio-7ecd436c7e33a740537fc0ef6fe9249b04f2fbe048ee9aa44dee154a288a05fd WatchSource:0}: Error finding container 7ecd436c7e33a740537fc0ef6fe9249b04f2fbe048ee9aa44dee154a288a05fd: Status 404 returned error can't find the container with id 7ecd436c7e33a740537fc0ef6fe9249b04f2fbe048ee9aa44dee154a288a05fd
Sep 15 21:17:00.825229 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.825202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:17:00.827056 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:17:00.827023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1407acf4_4f8d_4c32_a3f6_937aa9bc0fef.slice/crio-8f8c444edca0110fb8fe32f766483dd9cc9918bb34f5f03726f6b83dd8e8f904 WatchSource:0}: Error finding container 8f8c444edca0110fb8fe32f766483dd9cc9918bb34f5f03726f6b83dd8e8f904: Status 404 returned error can't find the container with id 8f8c444edca0110fb8fe32f766483dd9cc9918bb34f5f03726f6b83dd8e8f904
Sep 15 21:17:00.863468 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.863398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" event={"ID":"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef","Type":"ContainerStarted","Data":"8f8c444edca0110fb8fe32f766483dd9cc9918bb34f5f03726f6b83dd8e8f904"}
Sep 15 21:17:00.865454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:00.865395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" event={"ID":"993b6734-50fa-4b48-9f97-9603a313f472","Type":"ContainerStarted","Data":"7ecd436c7e33a740537fc0ef6fe9249b04f2fbe048ee9aa44dee154a288a05fd"}
Sep 15 21:17:01.874603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.874534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" event={"ID":"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef","Type":"ContainerStarted","Data":"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"}
Sep 15 21:17:01.875042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.874642    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:01.876336 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.876313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" event={"ID":"993b6734-50fa-4b48-9f97-9603a313f472","Type":"ContainerStarted","Data":"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"}
Sep 15 21:17:01.876488 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.876441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:01.899505 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.899454    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" podStartSLOduration=1.899436468 podStartE2EDuration="1.899436468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:01.898353318 +0000 UTC m=+1129.684721325" watchObservedRunningTime="2026-09-15 21:17:01.899436468 +0000 UTC m=+1129.685804504"
Sep 15 21:17:01.923791 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:01.923732    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" podStartSLOduration=1.923711534 podStartE2EDuration="1.923711534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:01.923156017 +0000 UTC m=+1129.709524026" watchObservedRunningTime="2026-09-15 21:17:01.923711534 +0000 UTC m=+1129.710079542"
Sep 15 21:17:12.790273 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:12.790241    2577 scope.go:117] "RemoveContainer" containerID="be06acadaa8cb698f543f975e3b9b9c574b8edf537c690324722a12ff29af40d"
Sep 15 21:17:12.885288 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:12.885262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:17:12.885433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:12.885339    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:17:30.061929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.061893    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"]
Sep 15 21:17:30.068268 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.068242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:30.071060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.071027    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:17:30.071216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.071027    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:17:30.075718 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.075695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"]
Sep 15 21:17:30.139871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.139830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j\" (UID: \"35162f71-c2dc-42bc-8f91-dd43215a3380\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:30.240942 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.240907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j\" (UID: \"35162f71-c2dc-42bc-8f91-dd43215a3380\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:30.241614 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.241552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j\" (UID: \"35162f71-c2dc-42bc-8f91-dd43215a3380\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:30.379829 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.379748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:30.663435 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:30.663408    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"]
Sep 15 21:17:30.665786 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:17:30.665751    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35162f71_c2dc_42bc_8f91_dd43215a3380.slice/crio-6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41 WatchSource:0}: Error finding container 6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41: Status 404 returned error can't find the container with id 6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41
Sep 15 21:17:31.071398 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:31.071365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j" event={"ID":"35162f71-c2dc-42bc-8f91-dd43215a3380","Type":"ContainerStarted","Data":"fc2007002b8e9b59955f90cfaab0e5de8dac06da79df97d098cc65e10a2c9bc4"}
Sep 15 21:17:31.071823 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:31.071422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j" event={"ID":"35162f71-c2dc-42bc-8f91-dd43215a3380","Type":"ContainerStarted","Data":"6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41"}
Sep 15 21:17:31.091754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:31.091711    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j" podStartSLOduration=1.091696098 podStartE2EDuration="1.091696098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:31.088679444 +0000 UTC m=+1158.875047452" watchObservedRunningTime="2026-09-15 21:17:31.091696098 +0000 UTC m=+1158.878064106"
Sep 15 21:17:32.079428 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:32.079392    2577 generic.go:356] "Generic (PLEG): container finished" podID="35162f71-c2dc-42bc-8f91-dd43215a3380" containerID="fc2007002b8e9b59955f90cfaab0e5de8dac06da79df97d098cc65e10a2c9bc4" exitCode=0
Sep 15 21:17:32.079840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:32.079465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j" event={"ID":"35162f71-c2dc-42bc-8f91-dd43215a3380","Type":"ContainerDied","Data":"fc2007002b8e9b59955f90cfaab0e5de8dac06da79df97d098cc65e10a2c9bc4"}
Sep 15 21:17:33.235007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:33.234979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:33.374755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:33.374671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca\") pod \"35162f71-c2dc-42bc-8f91-dd43215a3380\" (UID: \"35162f71-c2dc-42bc-8f91-dd43215a3380\") "
Sep 15 21:17:33.375028 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:33.375005    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "35162f71-c2dc-42bc-8f91-dd43215a3380" (UID: "35162f71-c2dc-42bc-8f91-dd43215a3380"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:33.475874 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:33.475846    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35162f71-c2dc-42bc-8f91-dd43215a3380-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:34.093309 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:34.093242    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"
Sep 15 21:17:34.093473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:34.093233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j" event={"ID":"35162f71-c2dc-42bc-8f91-dd43215a3380","Type":"ContainerDied","Data":"6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41"}
Sep 15 21:17:34.093473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:34.093380    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b5cd30d03969fdf57983f3bb185de27d285ce56d2c97cd58b110b11bb27cb41"
Sep 15 21:17:40.660775 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.660733    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:17:40.661236 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.660959    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" podUID="aae73e26-25cc-4ea2-b81d-b793de1ed273" containerName="payload-processing" containerID="cri-o://9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08" gracePeriod=30
Sep 15 21:17:40.671507 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.671477    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:17:40.671819 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.671788    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" podUID="805620bd-5e2a-415b-9105-0b576f7ce3f1" containerName="payload-pre-processing" containerID="cri-o://3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f" gracePeriod=30
Sep 15 21:17:40.863669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.863220    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"]
Sep 15 21:17:40.876956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.876699    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0df719-4qx5j"]
Sep 15 21:17:40.943147 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.943109    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:17:40.952131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.952103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr9wm\" (UniqueName: \"kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm\") pod \"aae73e26-25cc-4ea2-b81d-b793de1ed273\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") "
Sep 15 21:17:40.952312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.952299    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume\") pod \"aae73e26-25cc-4ea2-b81d-b793de1ed273\" (UID: \"aae73e26-25cc-4ea2-b81d-b793de1ed273\") "
Sep 15 21:17:40.952662 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.952638    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aae73e26-25cc-4ea2-b81d-b793de1ed273" (UID: "aae73e26-25cc-4ea2-b81d-b793de1ed273"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:40.954808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.954782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm" (OuterVolumeSpecName: "kube-api-access-mr9wm") pod "aae73e26-25cc-4ea2-b81d-b793de1ed273" (UID: "aae73e26-25cc-4ea2-b81d-b793de1ed273"). InnerVolumeSpecName "kube-api-access-mr9wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:40.988465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:40.988444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:17:41.053414 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.053374    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae73e26-25cc-4ea2-b81d-b793de1ed273-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.053633 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.053612    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr9wm\" (UniqueName: \"kubernetes.io/projected/aae73e26-25cc-4ea2-b81d-b793de1ed273-kube-api-access-mr9wm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.139502 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.139465    2577 generic.go:356] "Generic (PLEG): container finished" podID="aae73e26-25cc-4ea2-b81d-b793de1ed273" containerID="9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08" exitCode=0
Sep 15 21:17:41.139717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.139508    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"
Sep 15 21:17:41.139717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.139606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" event={"ID":"aae73e26-25cc-4ea2-b81d-b793de1ed273","Type":"ContainerDied","Data":"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"}
Sep 15 21:17:41.139717 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.139693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z" event={"ID":"aae73e26-25cc-4ea2-b81d-b793de1ed273","Type":"ContainerDied","Data":"eea15632818cf6f9f9e040202c2d6f4cb83ba0dc3fddcdcf1bdea9afa3929b17"}
Sep 15 21:17:41.139894 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.139730    2577 scope.go:117] "RemoveContainer" containerID="9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"
Sep 15 21:17:41.141141 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.141116    2577 generic.go:356] "Generic (PLEG): container finished" podID="805620bd-5e2a-415b-9105-0b576f7ce3f1" containerID="3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f" exitCode=0
Sep 15 21:17:41.141316 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.141151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" event={"ID":"805620bd-5e2a-415b-9105-0b576f7ce3f1","Type":"ContainerDied","Data":"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"}
Sep 15 21:17:41.141445 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.141359    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9" event={"ID":"805620bd-5e2a-415b-9105-0b576f7ce3f1","Type":"ContainerDied","Data":"16cdf6bfadd446c409c623a0fafe2707d529f91872b94d1a2468fe95853ccf9f"}
Sep 15 21:17:41.141445 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.141178    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"
Sep 15 21:17:41.150180 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.150164    2577 scope.go:117] "RemoveContainer" containerID="9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"
Sep 15 21:17:41.150455 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:17:41.150439    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08\": container with ID starting with 9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08 not found: ID does not exist" containerID="9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"
Sep 15 21:17:41.150502 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.150463    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08"} err="failed to get container status \"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08\": rpc error: code = NotFound desc = could not find container \"9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08\": container with ID starting with 9c944866bd8fdb0dc88f92d3e409d518e950d894563e7c3197e8293ee7ef6b08 not found: ID does not exist"
Sep 15 21:17:41.150502 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.150479    2577 scope.go:117] "RemoveContainer" containerID="3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"
Sep 15 21:17:41.153912 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.153894    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcmcq\" (UniqueName: \"kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq\") pod \"805620bd-5e2a-415b-9105-0b576f7ce3f1\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") "
Sep 15 21:17:41.154001 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.153952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume\") pod \"805620bd-5e2a-415b-9105-0b576f7ce3f1\" (UID: \"805620bd-5e2a-415b-9105-0b576f7ce3f1\") "
Sep 15 21:17:41.154340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.154317    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "805620bd-5e2a-415b-9105-0b576f7ce3f1" (UID: "805620bd-5e2a-415b-9105-0b576f7ce3f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:41.156206 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.156183    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq" (OuterVolumeSpecName: "kube-api-access-xcmcq") pod "805620bd-5e2a-415b-9105-0b576f7ce3f1" (UID: "805620bd-5e2a-415b-9105-0b576f7ce3f1"). InnerVolumeSpecName "kube-api-access-xcmcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:41.159389 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.159309    2577 scope.go:117] "RemoveContainer" containerID="3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"
Sep 15 21:17:41.160318 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:17:41.159781    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f\": container with ID starting with 3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f not found: ID does not exist" containerID="3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"
Sep 15 21:17:41.160318 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.159824    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f"} err="failed to get container status \"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f\": rpc error: code = NotFound desc = could not find container \"3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f\": container with ID starting with 3699ab73ef89c0c654a5d747da1eecdbe4ccfb387d46ffd5c17a6c168454670f not found: ID does not exist"
Sep 15 21:17:41.162902 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.162881    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:17:41.168344 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.168297    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0df719-766f6f6f79-5856z"]
Sep 15 21:17:41.254924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.254882    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805620bd-5e2a-415b-9105-0b576f7ce3f1-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.254924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.254921    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcmcq\" (UniqueName: \"kubernetes.io/projected/805620bd-5e2a-415b-9105-0b576f7ce3f1-kube-api-access-xcmcq\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.467515 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.467482    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:17:41.472195 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:41.472171    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0df719-75897f657d-gpvh9"]
Sep 15 21:17:42.691384 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:42.691351    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35162f71-c2dc-42bc-8f91-dd43215a3380" path="/var/lib/kubelet/pods/35162f71-c2dc-42bc-8f91-dd43215a3380/volumes"
Sep 15 21:17:42.691772 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:42.691743    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805620bd-5e2a-415b-9105-0b576f7ce3f1" path="/var/lib/kubelet/pods/805620bd-5e2a-415b-9105-0b576f7ce3f1/volumes"
Sep 15 21:17:42.692058 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:42.692046    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aae73e26-25cc-4ea2-b81d-b793de1ed273" path="/var/lib/kubelet/pods/aae73e26-25cc-4ea2-b81d-b793de1ed273/volumes"
Sep 15 21:17:47.889942 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:47.889906    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:17:47.890419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:47.890115    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy" containerID="cri-o://ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd" gracePeriod=30
Sep 15 21:17:48.221762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.221683    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"]
Sep 15 21:17:48.222188 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222174    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae73e26-25cc-4ea2-b81d-b793de1ed273" containerName="payload-processing"
Sep 15 21:17:48.222235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae73e26-25cc-4ea2-b81d-b793de1ed273" containerName="payload-processing"
Sep 15 21:17:48.222235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222216    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805620bd-5e2a-415b-9105-0b576f7ce3f1" containerName="payload-pre-processing"
Sep 15 21:17:48.222235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222222    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="805620bd-5e2a-415b-9105-0b576f7ce3f1" containerName="payload-pre-processing"
Sep 15 21:17:48.222235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222230    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35162f71-c2dc-42bc-8f91-dd43215a3380" containerName="revoke-keys"
Sep 15 21:17:48.222235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222235    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35162f71-c2dc-42bc-8f91-dd43215a3380" containerName="revoke-keys"
Sep 15 21:17:48.222392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222321    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35162f71-c2dc-42bc-8f91-dd43215a3380" containerName="revoke-keys"
Sep 15 21:17:48.222392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222331    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae73e26-25cc-4ea2-b81d-b793de1ed273" containerName="payload-processing"
Sep 15 21:17:48.222392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.222338    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="805620bd-5e2a-415b-9105-0b576f7ce3f1" containerName="payload-pre-processing"
Sep 15 21:17:48.226701 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.226682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:48.229313 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.229281    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:17:48.229447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.229335    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:17:48.235032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.235008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"]
Sep 15 21:17:48.322894 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.322860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg\" (UID: \"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:48.423726 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.423695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg\" (UID: \"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:48.424387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.424359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg\" (UID: \"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:48.540666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.540632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:48.820665 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:48.820625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"]
Sep 15 21:17:48.822359 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:17:48.822331    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf852f2_f7e8_46a5_bbed_6f05aa1d9bb6.slice/crio-da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2 WatchSource:0}: Error finding container da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2: Status 404 returned error can't find the container with id da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2
Sep 15 21:17:49.192165 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:49.192131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg" event={"ID":"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6","Type":"ContainerStarted","Data":"bcf554bfabb408a71ab9b2cc762615adb9a1ad4a9608c462f0858b6f3a38389d"}
Sep 15 21:17:49.192688 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:49.192189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg" event={"ID":"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6","Type":"ContainerStarted","Data":"da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2"}
Sep 15 21:17:49.208165 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:49.208116    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg" podStartSLOduration=1.208099445 podStartE2EDuration="1.208099445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:17:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:49.206750839 +0000 UTC m=+1176.993118840" watchObservedRunningTime="2026-09-15 21:17:49.208099445 +0000 UTC m=+1176.994467452"
Sep 15 21:17:50.199994 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:50.199960    2577 generic.go:356] "Generic (PLEG): container finished" podID="bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" containerID="bcf554bfabb408a71ab9b2cc762615adb9a1ad4a9608c462f0858b6f3a38389d" exitCode=0
Sep 15 21:17:50.200479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:50.200047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg" event={"ID":"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6","Type":"ContainerDied","Data":"bcf554bfabb408a71ab9b2cc762615adb9a1ad4a9608c462f0858b6f3a38389d"}
Sep 15 21:17:51.359422 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:51.359393    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:51.455997 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:51.455968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca\") pod \"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6\" (UID: \"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6\") "
Sep 15 21:17:51.456358 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:51.456331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" (UID: "bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:51.557864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:51.557827    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:52.221532 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:52.221504    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"
Sep 15 21:17:52.221532 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:52.221508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg" event={"ID":"bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6","Type":"ContainerDied","Data":"da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2"}
Sep 15 21:17:52.221757 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:52.221555    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da1cd164aad215256494885ca99b8004cb5d22245b8786f7f5250a276e97afe2"
Sep 15 21:17:53.156147 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.156086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:17:53.228765 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.228731    2577 generic.go:356] "Generic (PLEG): container finished" podID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerID="ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd" exitCode=0
Sep 15 21:17:53.228972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.228796    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"
Sep 15 21:17:53.228972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.228805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerDied","Data":"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"}
Sep 15 21:17:53.228972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.228858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh" event={"ID":"59ab8fba-290f-4b94-b3d2-ab3387f73c6a","Type":"ContainerDied","Data":"4861f0b678c1383189649e6fdfb2c6e00782c23d8673259bf7a7fbbea6e66a04"}
Sep 15 21:17:53.228972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.228892    2577 scope.go:117] "RemoveContainer" containerID="ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"
Sep 15 21:17:53.238894 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.238875    2577 scope.go:117] "RemoveContainer" containerID="db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"
Sep 15 21:17:53.247515 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.247497    2577 scope.go:117] "RemoveContainer" containerID="ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"
Sep 15 21:17:53.247832 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:17:53.247814    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd\": container with ID starting with ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd not found: ID does not exist" containerID="ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"
Sep 15 21:17:53.247923 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.247845    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd"} err="failed to get container status \"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd\": rpc error: code = NotFound desc = could not find container \"ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd\": container with ID starting with ad38474807c7fe407b2a5193302e33a74428e3caa7ffb1a7bf9c21d10e9a17fd not found: ID does not exist"
Sep 15 21:17:53.247923 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.247869    2577 scope.go:117] "RemoveContainer" containerID="db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"
Sep 15 21:17:53.248154 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:17:53.248137    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6\": container with ID starting with db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6 not found: ID does not exist" containerID="db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"
Sep 15 21:17:53.248191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.248162    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6"} err="failed to get container status \"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6\": rpc error: code = NotFound desc = could not find container \"db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6\": container with ID starting with db0ea7afcfd3064e1f64439f80c6d2268e6eb9974307661148d33aa247c52aa6 not found: ID does not exist"
Sep 15 21:17:53.274547 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274675 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274611    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274675 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274686    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274706    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m8k9\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274731    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.274790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.275040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket\") pod \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\" (UID: \"59ab8fba-290f-4b94-b3d2-ab3387f73c6a\") "
Sep 15 21:17:53.275040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:53.275040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.274956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:53.275200 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.275186    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.275253 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.275192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:53.275253 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.275206    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.275253 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.275211    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:53.275397 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.275308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:53.276990 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.276970    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:17:53.277210 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.277193    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9" (OuterVolumeSpecName: "kube-api-access-6m8k9") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "kube-api-access-6m8k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:53.277380 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.277361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:53.277437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.277389    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "59ab8fba-290f-4b94-b3d2-ab3387f73c6a" (UID: "59ab8fba-290f-4b94-b3d2-ab3387f73c6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:53.375989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.375957    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.375989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.375983    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.375989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.375993    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.376222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.376001    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m8k9\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-kube-api-access-6m8k9\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.376222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.376010    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.376222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.376019    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.376222 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.376026    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ab8fba-290f-4b94-b3d2-ab3387f73c6a-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:53.555375 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.555347    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:17:53.560967 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:53.560942    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-0df719-openshift-default-7fc8f45456-mtcmh"]
Sep 15 21:17:54.691693 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:54.691656    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" path="/var/lib/kubelet/pods/59ab8fba-290f-4b94-b3d2-ab3387f73c6a/volumes"
Sep 15 21:17:58.776476 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.776412    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"]
Sep 15 21:17:58.779312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.779290    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-253ca1-29825115-nv2zf"]
Sep 15 21:17:58.812450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.812410    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:17:58.812712 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.812676    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" podUID="9bcce461-5117-438b-81c1-f93e7bc188d1" containerName="payload-pre-processing" containerID="cri-o://34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34" gracePeriod=30
Sep 15 21:17:58.993097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.993058    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"]
Sep 15 21:17:58.997296 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:58.997271    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-253ca1-bfpdg"]
Sep 15 21:17:59.076220 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.076177    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:17:59.134495 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.134463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume\") pod \"9bcce461-5117-438b-81c1-f93e7bc188d1\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") "
Sep 15 21:17:59.134766 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.134507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4759\" (UniqueName: \"kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759\") pod \"9bcce461-5117-438b-81c1-f93e7bc188d1\" (UID: \"9bcce461-5117-438b-81c1-f93e7bc188d1\") "
Sep 15 21:17:59.135109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.134948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bcce461-5117-438b-81c1-f93e7bc188d1" (UID: "9bcce461-5117-438b-81c1-f93e7bc188d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:59.135109 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.135076    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcce461-5117-438b-81c1-f93e7bc188d1-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.137266 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.137231    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759" (OuterVolumeSpecName: "kube-api-access-l4759") pod "9bcce461-5117-438b-81c1-f93e7bc188d1" (UID: "9bcce461-5117-438b-81c1-f93e7bc188d1"). InnerVolumeSpecName "kube-api-access-l4759". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:59.236542 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.236511    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4759\" (UniqueName: \"kubernetes.io/projected/9bcce461-5117-438b-81c1-f93e7bc188d1-kube-api-access-l4759\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.274447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.274414    2577 generic.go:356] "Generic (PLEG): container finished" podID="9bcce461-5117-438b-81c1-f93e7bc188d1" containerID="34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34" exitCode=0
Sep 15 21:17:59.274603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.274460    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"
Sep 15 21:17:59.274603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.274504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" event={"ID":"9bcce461-5117-438b-81c1-f93e7bc188d1","Type":"ContainerDied","Data":"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"}
Sep 15 21:17:59.274603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.274557    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b" event={"ID":"9bcce461-5117-438b-81c1-f93e7bc188d1","Type":"ContainerDied","Data":"02a0f65edc8df2dfc02760b8332d6c834036c8fa6e6623fbfd163ee7443ddf87"}
Sep 15 21:17:59.274603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.274596    2577 scope.go:117] "RemoveContainer" containerID="34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"
Sep 15 21:17:59.284127 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.284105    2577 scope.go:117] "RemoveContainer" containerID="34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"
Sep 15 21:17:59.284412 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:17:59.284394    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34\": container with ID starting with 34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34 not found: ID does not exist" containerID="34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"
Sep 15 21:17:59.284468 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.284421    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34"} err="failed to get container status \"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34\": rpc error: code = NotFound desc = could not find container \"34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34\": container with ID starting with 34ec9950980c275afefed904b6d729ed85c9f1d3b65901f95247c80aae6ecd34 not found: ID does not exist"
Sep 15 21:17:59.297588 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.297543    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:17:59.303491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:17:59.303424    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-253ca1-98dcb56fd-l727b"]
Sep 15 21:18:00.691392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:00.691356    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bcce461-5117-438b-81c1-f93e7bc188d1" path="/var/lib/kubelet/pods/9bcce461-5117-438b-81c1-f93e7bc188d1/volumes"
Sep 15 21:18:00.691808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:00.691718    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" path="/var/lib/kubelet/pods/bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6/volumes"
Sep 15 21:18:00.691966 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:00.691953    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dccc42ce-33ca-45ed-a64b-cd91057d4927" path="/var/lib/kubelet/pods/dccc42ce-33ca-45ed-a64b-cd91057d4927/volumes"
Sep 15 21:18:12.707825 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:12.707803    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:18:12.710949 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:12.710927    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:18:12.821141 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:12.821102    2577 scope.go:117] "RemoveContainer" containerID="463a874757d814c9031b245a247f565df3ec0e83b3e5aa5cad28177605759df9"
Sep 15 21:18:14.011806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.011771    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:18:14.012244 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012231    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" containerName="revoke-keys"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012246    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" containerName="revoke-keys"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012258    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012264    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012271    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bcce461-5117-438b-81c1-f93e7bc188d1" containerName="payload-pre-processing"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012278    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bcce461-5117-438b-81c1-f93e7bc188d1" containerName="payload-pre-processing"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012286    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:14.012294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012291    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:14.012510 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012350    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:14.012510 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012359    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bcce461-5117-438b-81c1-f93e7bc188d1" containerName="payload-pre-processing"
Sep 15 21:18:14.012510 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.012368    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf852f2-f7e8-46a5-bbed-6f05aa1d9bb6" containerName="revoke-keys"
Sep 15 21:18:14.017069 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.017050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.019816 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.019797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2f34ef-dockercfg-z7g4n\""
Sep 15 21:18:14.019816 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.019810    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2f34ef\""
Sep 15 21:18:14.024455 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.024427    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:18:14.091865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.091832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg7bk\" (UniqueName: \"kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.092042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.091898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.193215 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.193170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg7bk\" (UniqueName: \"kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.193419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.193270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.194013 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.193993    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.202345 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.202317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg7bk\" (UniqueName: \"kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk\") pod \"payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.330130 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.330038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:14.605638 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:14.605612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:18:14.608136 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:18:14.608108    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3366106e_cdc7_4b50_b8f3_7e144439fc75.slice/crio-62a9fa4f41ef0f235608deaca134d559360de27e4fcbce8fc16e93ee4f1d7f40 WatchSource:0}: Error finding container 62a9fa4f41ef0f235608deaca134d559360de27e4fcbce8fc16e93ee4f1d7f40: Status 404 returned error can't find the container with id 62a9fa4f41ef0f235608deaca134d559360de27e4fcbce8fc16e93ee4f1d7f40
Sep 15 21:18:15.381821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:15.381787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" event={"ID":"3366106e-cdc7-4b50-b8f3-7e144439fc75","Type":"ContainerStarted","Data":"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"}
Sep 15 21:18:15.382409 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:15.381840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" event={"ID":"3366106e-cdc7-4b50-b8f3-7e144439fc75","Type":"ContainerStarted","Data":"62a9fa4f41ef0f235608deaca134d559360de27e4fcbce8fc16e93ee4f1d7f40"}
Sep 15 21:18:15.382409 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:15.381870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:15.409458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:15.409404    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" podStartSLOduration=2.409387805 podStartE2EDuration="2.409387805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:15.408093058 +0000 UTC m=+1203.194461067" watchObservedRunningTime="2026-09-15 21:18:15.409387805 +0000 UTC m=+1203.195755843"
Sep 15 21:18:26.390321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:26.390287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:18:40.110906 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.110874    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:18:40.111420 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.111406    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59ab8fba-290f-4b94-b3d2-ab3387f73c6a" containerName="istio-proxy"
Sep 15 21:18:40.114641 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.114617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.117347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.117330    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-41ffdf-openshift-default-dockercfg-mhrdm\""
Sep 15 21:18:40.132779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.132746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:18:40.259618 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.259976 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqlv8\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.260023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.259991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.260023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.260010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361532 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361532 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361491    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqlv8\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.361812 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.361776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.362186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.362029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.362186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.362054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.362186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.362157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.362300 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.362232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.362773 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.362753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.364189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.364156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.364296 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.364279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.370837 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.370808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.371225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.371178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqlv8\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8\") pod \"e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") " pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.425121 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.425088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:40.715426 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.715398    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:18:40.717487 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:18:40.717459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9d4cc2_8a28_4a34_b704_f8840fd2434b.slice/crio-950f3ee597ea79ca00a0994eaeca5158d6a23af69d2f68df4a40d17858ad2764 WatchSource:0}: Error finding container 950f3ee597ea79ca00a0994eaeca5158d6a23af69d2f68df4a40d17858ad2764: Status 404 returned error can't find the container with id 950f3ee597ea79ca00a0994eaeca5158d6a23af69d2f68df4a40d17858ad2764
Sep 15 21:18:40.719885 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.719844    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:18:40.719978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.719916    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:18:40.719978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:40.719946    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:18:41.542701 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:41.542666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerStarted","Data":"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"}
Sep 15 21:18:41.543098 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:41.542724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerStarted","Data":"950f3ee597ea79ca00a0994eaeca5158d6a23af69d2f68df4a40d17858ad2764"}
Sep 15 21:18:41.568631 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:41.568543    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podStartSLOduration=1.5685228439999999 podStartE2EDuration="1.568522844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:41.565892403 +0000 UTC m=+1229.352260436" watchObservedRunningTime="2026-09-15 21:18:41.568522844 +0000 UTC m=+1229.354890853"
Sep 15 21:18:42.425650 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:42.425613    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:42.431090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:42.431063    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:42.549325 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:42.549295    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:42.550294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:42.550277    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:18:48.876660 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.876621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:18:48.881034 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.881009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:48.884184 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.884164    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-41ffdf\""
Sep 15 21:18:48.896735 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.896711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:18:48.947850 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.947818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:48.948018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:48.947951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99sdw\" (UniqueName: \"kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.048590 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.048531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99sdw\" (UniqueName: \"kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.048766 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.048648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.049338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.049319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.060235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.060208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99sdw\" (UniqueName: \"kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw\") pod \"payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.192411 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.192323    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.472078 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.471974    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:18:49.475078 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:18:49.475051    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba393ea_9256_4fe7_8d60_a3e03e08e5ef.slice/crio-90e91dc444833b9181df98012ec3dff79d1eacf92621950ad3695e4d7bf51966 WatchSource:0}: Error finding container 90e91dc444833b9181df98012ec3dff79d1eacf92621950ad3695e4d7bf51966: Status 404 returned error can't find the container with id 90e91dc444833b9181df98012ec3dff79d1eacf92621950ad3695e4d7bf51966
Sep 15 21:18:49.593997 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.593960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" event={"ID":"bba393ea-9256-4fe7-8d60-a3e03e08e5ef","Type":"ContainerStarted","Data":"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"}
Sep 15 21:18:49.594165 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.594016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" event={"ID":"bba393ea-9256-4fe7-8d60-a3e03e08e5ef","Type":"ContainerStarted","Data":"90e91dc444833b9181df98012ec3dff79d1eacf92621950ad3695e4d7bf51966"}
Sep 15 21:18:49.594209 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.594157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:18:49.612512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:18:49.612457    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" podStartSLOduration=1.612440445 podStartE2EDuration="1.612440445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:49.609891837 +0000 UTC m=+1237.396259857" watchObservedRunningTime="2026-09-15 21:18:49.612440445 +0000 UTC m=+1237.398808517"
Sep 15 21:19:00.602748 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.602716    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:19:00.941315 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.941218    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"]
Sep 15 21:19:00.946191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.946161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:00.949193 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.948782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:19:00.949193 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.949030    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:19:00.954400 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:00.954145    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"]
Sep 15 21:19:01.071685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.071652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9\" (UID: \"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:01.172637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.172560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9\" (UID: \"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:01.173191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.173172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9\" (UID: \"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:01.268493 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.268457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:01.548629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.548517    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"]
Sep 15 21:19:01.676767 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:01.676726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9" event={"ID":"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed","Type":"ContainerStarted","Data":"3c76fa5974acc43dc45bdc360b946d573150cfff40fce5eb5df0653bb370c8f8"}
Sep 15 21:19:02.685513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:02.685418    2577 generic.go:356] "Generic (PLEG): container finished" podID="5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" containerID="afe131c3e09fb61cb6f0aff87f15ea49eb796f6be2091f1b021bd1d071612d32" exitCode=0
Sep 15 21:19:02.690407 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:02.690380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9" event={"ID":"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed","Type":"ContainerDied","Data":"afe131c3e09fb61cb6f0aff87f15ea49eb796f6be2091f1b021bd1d071612d32"}
Sep 15 21:19:03.855389 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:03.855363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:03.905283 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:03.905203    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca\") pod \"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed\" (UID: \"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed\") "
Sep 15 21:19:03.905632 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:03.905606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" (UID: "5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:19:04.006238 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:04.006189    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:19:04.712090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:04.712049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"
Sep 15 21:19:04.712090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:04.712071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9" event={"ID":"5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed","Type":"ContainerDied","Data":"3c76fa5974acc43dc45bdc360b946d573150cfff40fce5eb5df0653bb370c8f8"}
Sep 15 21:19:04.712332 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:04.712115    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c76fa5974acc43dc45bdc360b946d573150cfff40fce5eb5df0653bb370c8f8"
Sep 15 21:19:06.726649 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.726621    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerID="cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229" exitCode=0
Sep 15 21:19:06.727042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.726694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerDied","Data":"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"}
Sep 15 21:19:06.727132 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.727117    2577 scope.go:117] "RemoveContainer" containerID="cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"
Sep 15 21:19:06.728323 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.728292    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:19:06.728395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.728363    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:19:06.728437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:06.728411    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:19:07.752963 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:07.752652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerStarted","Data":"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"}
Sep 15 21:19:08.426349 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:08.426309    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:19:08.427672 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:08.427647    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 15 21:19:08.427759 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:08.427713    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 15 21:19:09.426209 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:09.426174    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 15 21:19:09.426637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:09.426239    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 15 21:19:10.425710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:10.425676    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 15 21:19:10.425935 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:10.425732    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 15 21:19:11.425733 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:11.425700    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 15 21:19:11.426189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:11.425754    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 15 21:19:12.426479 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:12.426442    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 15 21:19:12.427018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:12.426508    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 15 21:19:12.549963 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:12.549922    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:19:12.867409 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:12.867375    2577 scope.go:117] "RemoveContainer" containerID="3407f5cbb885032b15ed94332aba45656d5208b1017c5cc195d5bd83a385eed6"
Sep 15 21:19:12.877181 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:12.877159    2577 scope.go:117] "RemoveContainer" containerID="a538de9928f4e7c1b2674e9cef8b2077c91df11e74d9b550b3b24c04e7e7e076"
Sep 15 21:19:13.430299 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:13.430270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:19:13.431394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:13.431371    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:19:16.776327 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.776291    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:19:16.777105 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.777072    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" podUID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" containerName="payload-processing" containerID="cri-o://dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e" gracePeriod=30
Sep 15 21:19:16.789528 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.789499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:19:16.790023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.789963    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" podUID="993b6734-50fa-4b48-9f97-9603a313f472" containerName="payload-pre-processing" containerID="cri-o://d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992" gracePeriod=30
Sep 15 21:19:16.985647 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.985618    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"]
Sep 15 21:19:16.989191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:16.989167    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-56c6b012-6p5m9"]
Sep 15 21:19:17.051145 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.051116    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:19:17.075030 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.074951    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:19:17.154470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.153920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td9m6\" (UniqueName: \"kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6\") pod \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") "
Sep 15 21:19:17.154470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.154038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume\") pod \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\" (UID: \"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef\") "
Sep 15 21:19:17.155084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.154790    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" (UID: "1407acf4-4f8d-4c32-a3f6-937aa9bc0fef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:19:17.157392 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.157361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6" (OuterVolumeSpecName: "kube-api-access-td9m6") pod "1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" (UID: "1407acf4-4f8d-4c32-a3f6-937aa9bc0fef"). InnerVolumeSpecName "kube-api-access-td9m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:19:17.255677 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.255645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume\") pod \"993b6734-50fa-4b48-9f97-9603a313f472\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") "
Sep 15 21:19:17.255888 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.255743    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs24z\" (UniqueName: \"kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z\") pod \"993b6734-50fa-4b48-9f97-9603a313f472\" (UID: \"993b6734-50fa-4b48-9f97-9603a313f472\") "
Sep 15 21:19:17.255989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.255968    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td9m6\" (UniqueName: \"kubernetes.io/projected/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-kube-api-access-td9m6\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.255989 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.255983    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.256108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.256081    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "993b6734-50fa-4b48-9f97-9603a313f472" (UID: "993b6734-50fa-4b48-9f97-9603a313f472"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:19:17.257921 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.257900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z" (OuterVolumeSpecName: "kube-api-access-rs24z") pod "993b6734-50fa-4b48-9f97-9603a313f472" (UID: "993b6734-50fa-4b48-9f97-9603a313f472"). InnerVolumeSpecName "kube-api-access-rs24z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:19:17.357458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.357377    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993b6734-50fa-4b48-9f97-9603a313f472-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.357458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.357404    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs24z\" (UniqueName: \"kubernetes.io/projected/993b6734-50fa-4b48-9f97-9603a313f472-kube-api-access-rs24z\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.830766 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.830729    2577 generic.go:356] "Generic (PLEG): container finished" podID="993b6734-50fa-4b48-9f97-9603a313f472" containerID="d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992" exitCode=0
Sep 15 21:19:17.831217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.830829    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"
Sep 15 21:19:17.831217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.830912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" event={"ID":"993b6734-50fa-4b48-9f97-9603a313f472","Type":"ContainerDied","Data":"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"}
Sep 15 21:19:17.831217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.830954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j" event={"ID":"993b6734-50fa-4b48-9f97-9603a313f472","Type":"ContainerDied","Data":"7ecd436c7e33a740537fc0ef6fe9249b04f2fbe048ee9aa44dee154a288a05fd"}
Sep 15 21:19:17.831217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.830992    2577 scope.go:117] "RemoveContainer" containerID="d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"
Sep 15 21:19:17.835090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.834108    2577 generic.go:356] "Generic (PLEG): container finished" podID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" containerID="dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e" exitCode=0
Sep 15 21:19:17.835090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.834171    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"
Sep 15 21:19:17.835090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.834235    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" event={"ID":"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef","Type":"ContainerDied","Data":"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"}
Sep 15 21:19:17.835090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.834276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7" event={"ID":"1407acf4-4f8d-4c32-a3f6-937aa9bc0fef","Type":"ContainerDied","Data":"8f8c444edca0110fb8fe32f766483dd9cc9918bb34f5f03726f6b83dd8e8f904"}
Sep 15 21:19:17.843353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.843067    2577 scope.go:117] "RemoveContainer" containerID="d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"
Sep 15 21:19:17.843524 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:19:17.843503    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992\": container with ID starting with d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992 not found: ID does not exist" containerID="d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"
Sep 15 21:19:17.843637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.843532    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992"} err="failed to get container status \"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992\": rpc error: code = NotFound desc = could not find container \"d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992\": container with ID starting with d6b2a895d89cfe27918cb67bfaabdff8db8d220fb7126bda9eeac32637763992 not found: ID does not exist"
Sep 15 21:19:17.843637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.843547    2577 scope.go:117] "RemoveContainer" containerID="dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"
Sep 15 21:19:17.854949 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.854931    2577 scope.go:117] "RemoveContainer" containerID="dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"
Sep 15 21:19:17.855242 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:19:17.855222    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e\": container with ID starting with dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e not found: ID does not exist" containerID="dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"
Sep 15 21:19:17.855312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.855251    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e"} err="failed to get container status \"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e\": rpc error: code = NotFound desc = could not find container \"dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e\": container with ID starting with dbfe8633985351b80b1e03822f3a899b58d4b32a9a0650044c5649a4ea12ed2e not found: ID does not exist"
Sep 15 21:19:17.860427 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.860398    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:19:17.868023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.866464    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-56c6b012-658f6f6897-kjfv7"]
Sep 15 21:19:17.881341 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.881301    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:19:17.891162 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:17.891139    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-56c6b012-664bc996c8-f659j"]
Sep 15 21:19:18.690908 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:18.690871    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" path="/var/lib/kubelet/pods/1407acf4-4f8d-4c32-a3f6-937aa9bc0fef/volumes"
Sep 15 21:19:18.691215 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:18.691202    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" path="/var/lib/kubelet/pods/5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed/volumes"
Sep 15 21:19:18.691512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:19:18.691501    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993b6734-50fa-4b48-9f97-9603a313f472" path="/var/lib/kubelet/pods/993b6734-50fa-4b48-9f97-9603a313f472/volumes"
Sep 15 21:20:12.910012 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:20:12.909983    2577 scope.go:117] "RemoveContainer" containerID="217ece53f17c8855d0b8c90c973ea19ca5b14ef9980e66be89f39106acc02659"
Sep 15 21:20:12.921417 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:20:12.921397    2577 scope.go:117] "RemoveContainer" containerID="0c42445a0abf2b95669837725e7a959c0d215363d091cda843bea3c4cb80c495"
Sep 15 21:21:08.298400 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.298360    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:21:08.299119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299097    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" containerName="revoke-keys"
Sep 15 21:21:08.299212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299122    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" containerName="revoke-keys"
Sep 15 21:21:08.299212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299140    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993b6734-50fa-4b48-9f97-9603a313f472" containerName="payload-pre-processing"
Sep 15 21:21:08.299212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299148    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="993b6734-50fa-4b48-9f97-9603a313f472" containerName="payload-pre-processing"
Sep 15 21:21:08.299212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299170    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" containerName="payload-processing"
Sep 15 21:21:08.299212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299178    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" containerName="payload-processing"
Sep 15 21:21:08.299465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299285    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="993b6734-50fa-4b48-9f97-9603a313f472" containerName="payload-pre-processing"
Sep 15 21:21:08.299465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299296    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be3c9bf-76f0-4a72-8a42-eb14afe7c5ed" containerName="revoke-keys"
Sep 15 21:21:08.299465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.299310    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1407acf4-4f8d-4c32-a3f6-937aa9bc0fef" containerName="payload-processing"
Sep 15 21:21:08.302156 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.302136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.304816 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.304796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8f89be-openshift-default-dockercfg-hthxt\""
Sep 15 21:21:08.313593 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.313545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.462710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.462823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4966\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.462905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.462985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.463151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.463209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.463247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.463283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.463808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.463340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.564831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4966\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.565053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.565053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.564913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.565239 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.565219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.565315 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.565280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.565502 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.565481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.567692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.567662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.567834 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.567782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.573371 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.573348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.573751 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.573731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4966\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966\") pod \"e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") " pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.616956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.616926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:08.896699 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.896673    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:21:08.898329 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:21:08.898302    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37a4aaf_9cf0_4691_b23d_8fe47de2d1bf.slice/crio-ce9341d8a391cf74f5aca2619ec9f5a5a652092640933ae1ff51acd8ee0757d9 WatchSource:0}: Error finding container ce9341d8a391cf74f5aca2619ec9f5a5a652092640933ae1ff51acd8ee0757d9: Status 404 returned error can't find the container with id ce9341d8a391cf74f5aca2619ec9f5a5a652092640933ae1ff51acd8ee0757d9
Sep 15 21:21:08.900755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.900720    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:08.900836 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.900794    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:08.900836 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:08.900828    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:09.514554 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:09.514516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerStarted","Data":"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"}
Sep 15 21:21:09.514965 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:09.514606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerStarted","Data":"ce9341d8a391cf74f5aca2619ec9f5a5a652092640933ae1ff51acd8ee0757d9"}
Sep 15 21:21:09.538261 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:09.538202    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podStartSLOduration=1.5381837219999999 podStartE2EDuration="1.538183722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:09.535905188 +0000 UTC m=+1377.322273210" watchObservedRunningTime="2026-09-15 21:21:09.538183722 +0000 UTC m=+1377.324551730"
Sep 15 21:21:09.617613 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:09.617586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:09.622462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:09.622439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:10.520582 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:10.520523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:10.521702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:10.521683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:12.955428 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:12.955398    2577 scope.go:117] "RemoveContainer" containerID="da313b4d66cf3f3d0b86109d59881fdd034ad421ee4a3998e14d9c38f626d07a"
Sep 15 21:21:12.964326 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:12.964297    2577 scope.go:117] "RemoveContainer" containerID="e0a89d4d9d14bfaf33c5481cdb1b26cde843fb7e9fff6196f8095f8c29b82ea8"
Sep 15 21:21:12.972404 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:12.972384    2577 scope.go:117] "RemoveContainer" containerID="598e91cac1156d0b7cce306cb5f2950e2068fc6db2cb563ce182bf4271d93769"
Sep 15 21:21:12.981557 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:12.981528    2577 scope.go:117] "RemoveContainer" containerID="1cbe0f8d1f8d132d1f4be07bacad0956d433c5da39c8eed0310f8f8e28e1aaf2"
Sep 15 21:21:16.269186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.269156    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:21:16.274381 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.274364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.277114 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.277089    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-8f89be\""
Sep 15 21:21:16.277335 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.277320    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-8f89be-dockercfg-67rpp\""
Sep 15 21:21:16.282777 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.282194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:21:16.312063 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.312031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:21:16.316426 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.316408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.322923 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.322898    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:21:16.347007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.346974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.347159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.347055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.347159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.347089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzlf\" (UniqueName: \"kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.347159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.347123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgjw\" (UniqueName: \"kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.448637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.448599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.448823 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.448671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.448823 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.448698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzlf\" (UniqueName: \"kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.448823 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.448713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgjw\" (UniqueName: \"kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.449320 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.449296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.449385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.449346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.456941 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.456915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgjw\" (UniqueName: \"kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw\") pod \"payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.457049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.456969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzlf\" (UniqueName: \"kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf\") pod \"payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.586291 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.586213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:16.630409 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.630166    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:16.873272 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.873238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:21:16.873625 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:21:16.873601    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69109814_6691_471e_bab0_1ba74fe84615.slice/crio-4f18ce7eb634329119400648936a6facb4a59da59d47ac0fa64bb2551a60750e WatchSource:0}: Error finding container 4f18ce7eb634329119400648936a6facb4a59da59d47ac0fa64bb2551a60750e: Status 404 returned error can't find the container with id 4f18ce7eb634329119400648936a6facb4a59da59d47ac0fa64bb2551a60750e
Sep 15 21:21:16.935810 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:16.935776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:21:17.565285 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.565244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" event={"ID":"cf6df347-002d-4a10-aba7-5f5ac6b33b6f","Type":"ContainerStarted","Data":"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"}
Sep 15 21:21:17.565768 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.565312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" event={"ID":"cf6df347-002d-4a10-aba7-5f5ac6b33b6f","Type":"ContainerStarted","Data":"580c86887369e5ec74f5902256266f5e9aac815c6d0f95c74023a3353b9ee94b"}
Sep 15 21:21:17.565768 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.565341    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:17.567622 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.567590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" event={"ID":"69109814-6691-471e-bab0-1ba74fe84615","Type":"ContainerStarted","Data":"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"}
Sep 15 21:21:17.567776 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.567647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:17.567776 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.567670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" event={"ID":"69109814-6691-471e-bab0-1ba74fe84615","Type":"ContainerStarted","Data":"4f18ce7eb634329119400648936a6facb4a59da59d47ac0fa64bb2551a60750e"}
Sep 15 21:21:17.586888 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.586844    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" podStartSLOduration=1.5868287300000001 podStartE2EDuration="1.58682873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:17.586151206 +0000 UTC m=+1385.372519213" watchObservedRunningTime="2026-09-15 21:21:17.58682873 +0000 UTC m=+1385.373196739"
Sep 15 21:21:17.610867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:17.610823    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" podStartSLOduration=1.610808718 podStartE2EDuration="1.610808718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:17.609411415 +0000 UTC m=+1385.395779423" watchObservedRunningTime="2026-09-15 21:21:17.610808718 +0000 UTC m=+1385.397176770"
Sep 15 21:21:19.386614 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.386583    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"]
Sep 15 21:21:19.390361 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.390345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:19.393100 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.393074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:21:19.393231 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.393114    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:21:19.397992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.397970    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"]
Sep 15 21:21:19.482526 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.482494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-544890-x27p2\" (UID: \"2630c315-31d3-4772-b86b-132575960d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:19.583784 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.583752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-544890-x27p2\" (UID: \"2630c315-31d3-4772-b86b-132575960d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:19.584337 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.584317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-544890-x27p2\" (UID: \"2630c315-31d3-4772-b86b-132575960d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:19.703081 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.703007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:19.976530 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:19.976506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"]
Sep 15 21:21:19.978411 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:21:19.978385    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2630c315_31d3_4772_b86b_132575960d5c.slice/crio-8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660 WatchSource:0}: Error finding container 8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660: Status 404 returned error can't find the container with id 8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660
Sep 15 21:21:20.589829 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:20.589729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2" event={"ID":"2630c315-31d3-4772-b86b-132575960d5c","Type":"ContainerStarted","Data":"de2949f1861578de69e64052a068535b7abf5e038f122e7b263e982af396f074"}
Sep 15 21:21:20.590230 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:20.589832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2" event={"ID":"2630c315-31d3-4772-b86b-132575960d5c","Type":"ContainerStarted","Data":"8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660"}
Sep 15 21:21:20.607660 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:20.607610    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2" podStartSLOduration=1.6075949920000001 podStartE2EDuration="1.607594992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:20.605233415 +0000 UTC m=+1388.391601424" watchObservedRunningTime="2026-09-15 21:21:20.607594992 +0000 UTC m=+1388.393963000"
Sep 15 21:21:21.597425 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:21.597390    2577 generic.go:356] "Generic (PLEG): container finished" podID="2630c315-31d3-4772-b86b-132575960d5c" containerID="de2949f1861578de69e64052a068535b7abf5e038f122e7b263e982af396f074" exitCode=0
Sep 15 21:21:21.597892 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:21.597471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2" event={"ID":"2630c315-31d3-4772-b86b-132575960d5c","Type":"ContainerDied","Data":"de2949f1861578de69e64052a068535b7abf5e038f122e7b263e982af396f074"}
Sep 15 21:21:22.752896 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:22.752875    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:22.817895 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:22.817861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca\") pod \"2630c315-31d3-4772-b86b-132575960d5c\" (UID: \"2630c315-31d3-4772-b86b-132575960d5c\") "
Sep 15 21:21:22.818209 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:22.818180    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2630c315-31d3-4772-b86b-132575960d5c" (UID: "2630c315-31d3-4772-b86b-132575960d5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:21:22.918876 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:22.918801    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2630c315-31d3-4772-b86b-132575960d5c-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:23.611301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:23.611263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2" event={"ID":"2630c315-31d3-4772-b86b-132575960d5c","Type":"ContainerDied","Data":"8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660"}
Sep 15 21:21:23.611478 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:23.611319    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bc5c0f58693cf883d170b7ddf9e25dff6cd9467657d8587ab15e8f108a7e660"
Sep 15 21:21:23.611478 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:23.611279    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"
Sep 15 21:21:24.283893 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:24.283860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:21:28.577232 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:28.577202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:21:28.577743 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:28.577283    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:21:35.180097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:35.179996    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"]
Sep 15 21:21:35.183250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:35.183228    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-544890-x27p2"]
Sep 15 21:21:36.693037 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:36.692995    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2630c315-31d3-4772-b86b-132575960d5c" path="/var/lib/kubelet/pods/2630c315-31d3-4772-b86b-132575960d5c/volumes"
Sep 15 21:21:38.728509 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.728475    2577 generic.go:356] "Generic (PLEG): container finished" podID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerID="034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab" exitCode=0
Sep 15 21:21:38.729028 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.728545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerDied","Data":"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"}
Sep 15 21:21:38.729079 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.729054    2577 scope.go:117] "RemoveContainer" containerID="034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"
Sep 15 21:21:38.730371 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.730338    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:38.730499 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.730416    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:38.730499 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:38.730455    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:39.305728 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.305681    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" podUID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" containerName="main" containerID="cri-o://a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300" gracePeriod=45
Sep 15 21:21:39.547645 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.547623    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:21:39.695272 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695182    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695429 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695429 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695429 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695402    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695503    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t8mf\" (UniqueName: \"kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf\") pod \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\" (UID: \"daedcfa9-b116-44bd-9dc6-8baf87f33d13\") "
Sep 15 21:21:39.695552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695487    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.695729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home" (OuterVolumeSpecName: "home") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.695790 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.695772    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache" (OuterVolumeSpecName: "model-cache") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.696401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.696336    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-home\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.696401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.696363    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-model-cache\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.696401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.696379    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tmp-dir\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.698139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.698098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf" (OuterVolumeSpecName: "kube-api-access-9t8mf") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "kube-api-access-9t8mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:39.698250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.698165    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:39.698250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.698167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm" (OuterVolumeSpecName: "dshm") pod "daedcfa9-b116-44bd-9dc6-8baf87f33d13" (UID: "daedcfa9-b116-44bd-9dc6-8baf87f33d13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.738962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.738930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerStarted","Data":"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"}
Sep 15 21:21:39.740724 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.740693    2577 generic.go:356] "Generic (PLEG): container finished" podID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" containerID="a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300" exitCode=0
Sep 15 21:21:39.740869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.740738    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"
Sep 15 21:21:39.740869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.740773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" event={"ID":"daedcfa9-b116-44bd-9dc6-8baf87f33d13","Type":"ContainerDied","Data":"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"}
Sep 15 21:21:39.740869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.740810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm" event={"ID":"daedcfa9-b116-44bd-9dc6-8baf87f33d13","Type":"ContainerDied","Data":"31658fb926e4c367eff74a802558d1885c7328acf5d5ad32ef1b8f0ab37ce117"}
Sep 15 21:21:39.740869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.740830    2577 scope.go:117] "RemoveContainer" containerID="a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"
Sep 15 21:21:39.752312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.752294    2577 scope.go:117] "RemoveContainer" containerID="a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"
Sep 15 21:21:39.752657 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:21:39.752635    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300\": container with ID starting with a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300 not found: ID does not exist" containerID="a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"
Sep 15 21:21:39.752738 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.752665    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300"} err="failed to get container status \"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300\": rpc error: code = NotFound desc = could not find container \"a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300\": container with ID starting with a958db79c9af95af2c3427b71255ef9b2224fc2440454c8783c029eddb6c8300 not found: ID does not exist"
Sep 15 21:21:39.797420 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.797392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t8mf\" (UniqueName: \"kubernetes.io/projected/daedcfa9-b116-44bd-9dc6-8baf87f33d13-kube-api-access-9t8mf\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.797558 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.797424    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daedcfa9-b116-44bd-9dc6-8baf87f33d13-tls-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.797558 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.797439    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daedcfa9-b116-44bd-9dc6-8baf87f33d13-dshm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.819119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.819094    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:21:39.821344 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:39.821325    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-544890/premium-w0-544890-kserve-78ff654988-sh8fm"]
Sep 15 21:21:40.521032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:40.520995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:40.617540 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:40.617508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:40.618806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:40.618780    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:40.618918 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:40.618846    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:40.692504 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:40.692473    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" path="/var/lib/kubelet/pods/daedcfa9-b116-44bd-9dc6-8baf87f33d13/volumes"
Sep 15 21:21:41.617670 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:41.617635    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:41.618031 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:41.617693    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:42.224484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.224446    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:21:42.224962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.224948    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" containerName="main"
Sep 15 21:21:42.225040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.224964    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" containerName="main"
Sep 15 21:21:42.225040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.224979    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2630c315-31d3-4772-b86b-132575960d5c" containerName="revoke-keys"
Sep 15 21:21:42.225040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.224986    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2630c315-31d3-4772-b86b-132575960d5c" containerName="revoke-keys"
Sep 15 21:21:42.225207 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.225060    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="daedcfa9-b116-44bd-9dc6-8baf87f33d13" containerName="main"
Sep 15 21:21:42.225207 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.225070    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2630c315-31d3-4772-b86b-132575960d5c" containerName="revoke-keys"
Sep 15 21:21:42.228318 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.228290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.245609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.241940    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-beb398-openshift-default-dockercfg-ttlg7\""
Sep 15 21:21:42.250592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.247655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:21:42.324367 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmj6l\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324826 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324826 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324826 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.324826 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.324775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.425885 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.425846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.425921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmj6l\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.425966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.426824 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.427050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.426976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.427050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.427002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.427050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.427039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.428518 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.428496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.428710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.428681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.434328 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.434294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.443286 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.443231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmj6l\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l\") pod \"e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") " pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.573301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.573264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:42.617938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.617904    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:42.618396 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.617968    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:42.869627 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.869540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:21:42.871390 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:21:42.871364    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d8521d_6f8f_470a_9bc3_3bcd0a300953.slice/crio-b65711c2997a95df27c876d69cf26eecda583499a2ec5819ebd832be908416fb WatchSource:0}: Error finding container b65711c2997a95df27c876d69cf26eecda583499a2ec5819ebd832be908416fb: Status 404 returned error can't find the container with id b65711c2997a95df27c876d69cf26eecda583499a2ec5819ebd832be908416fb
Sep 15 21:21:42.873867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.873834    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:42.873948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.873903    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:42.873948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:42.873932    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:21:43.618264 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:43.618230    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:43.618707 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:43.618301    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:43.775083 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:43.775041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerStarted","Data":"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"}
Sep 15 21:21:43.775280 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:43.775123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerStarted","Data":"b65711c2997a95df27c876d69cf26eecda583499a2ec5819ebd832be908416fb"}
Sep 15 21:21:43.795204 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:43.795152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podStartSLOduration=1.7951364920000001 podStartE2EDuration="1.795136492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:43.793899891 +0000 UTC m=+1411.580267899" watchObservedRunningTime="2026-09-15 21:21:43.795136492 +0000 UTC m=+1411.581504499"
Sep 15 21:21:44.573901 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.573862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:44.579321 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.579294    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:44.618285 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.618255    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:44.618676 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.618306    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:44.780441 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.780406    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:44.781486 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:44.781463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:21:45.617667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:45.617630    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 15 21:21:45.617827 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:45.617696    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 15 21:21:46.621726 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:46.621691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:21:46.622756 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:21:46.622735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:22:09.940247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.940152    2577 generic.go:356] "Generic (PLEG): container finished" podID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerID="1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23" exitCode=0
Sep 15 21:22:09.940247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.940208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerDied","Data":"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"}
Sep 15 21:22:09.940801 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.940615    2577 scope.go:117] "RemoveContainer" containerID="1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"
Sep 15 21:22:09.941925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.941894    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:22:09.941996 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.941968    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:22:09.942039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:09.942001    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:22:10.953244 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:10.953196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerStarted","Data":"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"}
Sep 15 21:22:11.573629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:11.573591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:22:11.574632 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:11.574598    2577 patch_prober.go:28] interesting pod/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 15 21:22:11.574752 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:11.574650    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 15 21:22:12.574196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:12.574164    2577 patch_prober.go:28] interesting pod/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 15 21:22:12.574604 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:12.574216    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 15 21:22:13.030042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:13.030002    2577 scope.go:117] "RemoveContainer" containerID="fbf360d3cde1582e445f8bf3780330e630453c73bd5397624152938a5ece9ae4"
Sep 15 21:22:13.573667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:13.573622    2577 patch_prober.go:28] interesting pod/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 15 21:22:13.573878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:13.573680    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 15 21:22:14.574529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:14.574485    2577 patch_prober.go:28] interesting pod/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 15 21:22:14.574967 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:14.574552    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 15 21:22:14.780618 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:14.780554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:22:15.574393 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:15.574356    2577 patch_prober.go:28] interesting pod/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 15 21:22:15.574617 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:15.574428    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 15 21:22:16.578277 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:16.578246    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:22:16.579475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:16.579454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:22:18.810241 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.810205    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"]
Sep 15 21:22:18.815307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.815288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:18.818488 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.818438    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:22:18.819502 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.819483    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:22:18.822875 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.822851    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"]
Sep 15 21:22:18.836054 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.832373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8f89be-xh46r\" (UID: \"09698a67-bfc2-4cbf-82dc-e92ae2fad011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:18.933467 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.933415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8f89be-xh46r\" (UID: \"09698a67-bfc2-4cbf-82dc-e92ae2fad011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:18.934204 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:18.934184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8f89be-xh46r\" (UID: \"09698a67-bfc2-4cbf-82dc-e92ae2fad011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:19.127962 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:19.127873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:19.406394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:19.406357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"]
Sep 15 21:22:19.409008 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:22:19.408982    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09698a67_bfc2_4cbf_82dc_e92ae2fad011.slice/crio-11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40 WatchSource:0}: Error finding container 11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40: Status 404 returned error can't find the container with id 11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40
Sep 15 21:22:20.013654 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:20.013547    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r" event={"ID":"09698a67-bfc2-4cbf-82dc-e92ae2fad011","Type":"ContainerStarted","Data":"1a2c08117bd553e6fc0f356d098cab8fd5fa262c9ee5280c0a408e2ba4917a8e"}
Sep 15 21:22:20.013654 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:20.013624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r" event={"ID":"09698a67-bfc2-4cbf-82dc-e92ae2fad011","Type":"ContainerStarted","Data":"11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40"}
Sep 15 21:22:20.059340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:20.059265    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r" podStartSLOduration=2.059244333 podStartE2EDuration="2.059244333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:22:20.057175491 +0000 UTC m=+1447.843543500" watchObservedRunningTime="2026-09-15 21:22:20.059244333 +0000 UTC m=+1447.845612344"
Sep 15 21:22:21.020501 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:21.020469    2577 generic.go:356] "Generic (PLEG): container finished" podID="09698a67-bfc2-4cbf-82dc-e92ae2fad011" containerID="1a2c08117bd553e6fc0f356d098cab8fd5fa262c9ee5280c0a408e2ba4917a8e" exitCode=0
Sep 15 21:22:21.020911 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:21.020553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r" event={"ID":"09698a67-bfc2-4cbf-82dc-e92ae2fad011","Type":"ContainerDied","Data":"1a2c08117bd553e6fc0f356d098cab8fd5fa262c9ee5280c0a408e2ba4917a8e"}
Sep 15 21:22:22.168995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:22.168971    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:22.268961 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:22.268926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca\") pod \"09698a67-bfc2-4cbf-82dc-e92ae2fad011\" (UID: \"09698a67-bfc2-4cbf-82dc-e92ae2fad011\") "
Sep 15 21:22:22.269243 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:22.269222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09698a67-bfc2-4cbf-82dc-e92ae2fad011" (UID: "09698a67-bfc2-4cbf-82dc-e92ae2fad011"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:22.369994 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:22.369913    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09698a67-bfc2-4cbf-82dc-e92ae2fad011-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:23.034236 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:23.034201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r" event={"ID":"09698a67-bfc2-4cbf-82dc-e92ae2fad011","Type":"ContainerDied","Data":"11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40"}
Sep 15 21:22:23.034236 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:23.034240    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"
Sep 15 21:22:23.034503 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:23.034252    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11669c67438e2c08525d4624ad003a03d656e4f42367506b888ba31260662e40"
Sep 15 21:22:34.598229 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.598185    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:22:34.598742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.598403    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" podUID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" containerName="payload-processing" containerID="cri-o://e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68" gracePeriod=30
Sep 15 21:22:34.607163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.607133    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:22:34.607413 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.607365    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" podUID="69109814-6691-471e-bab0-1ba74fe84615" containerName="payload-pre-processing" containerID="cri-o://d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159" gracePeriod=30
Sep 15 21:22:34.799372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.799338    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"]
Sep 15 21:22:34.801900 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.801880    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8f89be-xh46r"]
Sep 15 21:22:34.906667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.906639    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:22:34.909731 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:34.909715    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:22:35.009197 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009161    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume\") pod \"69109814-6691-471e-bab0-1ba74fe84615\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") "
Sep 15 21:22:35.009404 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009225    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjgjw\" (UniqueName: \"kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw\") pod \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") "
Sep 15 21:22:35.009404 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009311    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume\") pod \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\" (UID: \"cf6df347-002d-4a10-aba7-5f5ac6b33b6f\") "
Sep 15 21:22:35.009543 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pzlf\" (UniqueName: \"kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf\") pod \"69109814-6691-471e-bab0-1ba74fe84615\" (UID: \"69109814-6691-471e-bab0-1ba74fe84615\") "
Sep 15 21:22:35.009664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009636    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69109814-6691-471e-bab0-1ba74fe84615" (UID: "69109814-6691-471e-bab0-1ba74fe84615"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:35.009762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009736    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf6df347-002d-4a10-aba7-5f5ac6b33b6f" (UID: "cf6df347-002d-4a10-aba7-5f5ac6b33b6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:35.009893 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009879    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:35.009956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.009901    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69109814-6691-471e-bab0-1ba74fe84615-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:35.011898 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.011874    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf" (OuterVolumeSpecName: "kube-api-access-7pzlf") pod "69109814-6691-471e-bab0-1ba74fe84615" (UID: "69109814-6691-471e-bab0-1ba74fe84615"). InnerVolumeSpecName "kube-api-access-7pzlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:35.012536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.012503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw" (OuterVolumeSpecName: "kube-api-access-sjgjw") pod "cf6df347-002d-4a10-aba7-5f5ac6b33b6f" (UID: "cf6df347-002d-4a10-aba7-5f5ac6b33b6f"). InnerVolumeSpecName "kube-api-access-sjgjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:35.106013 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.105974    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" containerID="e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68" exitCode=0
Sep 15 21:22:35.106216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.106025    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"
Sep 15 21:22:35.106216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.106057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" event={"ID":"cf6df347-002d-4a10-aba7-5f5ac6b33b6f","Type":"ContainerDied","Data":"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"}
Sep 15 21:22:35.106216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.106119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w" event={"ID":"cf6df347-002d-4a10-aba7-5f5ac6b33b6f","Type":"ContainerDied","Data":"580c86887369e5ec74f5902256266f5e9aac815c6d0f95c74023a3353b9ee94b"}
Sep 15 21:22:35.106216 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.106149    2577 scope.go:117] "RemoveContainer" containerID="e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"
Sep 15 21:22:35.107840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.107811    2577 generic.go:356] "Generic (PLEG): container finished" podID="69109814-6691-471e-bab0-1ba74fe84615" containerID="d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159" exitCode=0
Sep 15 21:22:35.107992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.107854    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"
Sep 15 21:22:35.107992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.107886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" event={"ID":"69109814-6691-471e-bab0-1ba74fe84615","Type":"ContainerDied","Data":"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"}
Sep 15 21:22:35.107992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.107930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h" event={"ID":"69109814-6691-471e-bab0-1ba74fe84615","Type":"ContainerDied","Data":"4f18ce7eb634329119400648936a6facb4a59da59d47ac0fa64bb2551a60750e"}
Sep 15 21:22:35.111147 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.111040    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pzlf\" (UniqueName: \"kubernetes.io/projected/69109814-6691-471e-bab0-1ba74fe84615-kube-api-access-7pzlf\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:35.111147 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.111069    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjgjw\" (UniqueName: \"kubernetes.io/projected/cf6df347-002d-4a10-aba7-5f5ac6b33b6f-kube-api-access-sjgjw\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:35.119062 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.119028    2577 scope.go:117] "RemoveContainer" containerID="e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"
Sep 15 21:22:35.119389 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:22:35.119361    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68\": container with ID starting with e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68 not found: ID does not exist" containerID="e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"
Sep 15 21:22:35.119471 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.119398    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68"} err="failed to get container status \"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68\": rpc error: code = NotFound desc = could not find container \"e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68\": container with ID starting with e5100405e25f46cdcded33f524fb2ec025401053b23ef760d7703f8596a42a68 not found: ID does not exist"
Sep 15 21:22:35.119471 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.119423    2577 scope.go:117] "RemoveContainer" containerID="d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"
Sep 15 21:22:35.129818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.128743    2577 scope.go:117] "RemoveContainer" containerID="d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"
Sep 15 21:22:35.129818 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:22:35.129218    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159\": container with ID starting with d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159 not found: ID does not exist" containerID="d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"
Sep 15 21:22:35.129818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.129281    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159"} err="failed to get container status \"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159\": rpc error: code = NotFound desc = could not find container \"d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159\": container with ID starting with d86e13315787c7b05ce8d41d8a3d308a4a93297621e22dc90b526249787e2159 not found: ID does not exist"
Sep 15 21:22:35.138121 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.138064    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:22:35.143638 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.143618    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f89be-7f8d764d79-m478w"]
Sep 15 21:22:35.157514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.157485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:22:35.163939 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:35.163914    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f89be-74b8bc689d-mkc7h"]
Sep 15 21:22:36.691491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:36.691461    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09698a67-bfc2-4cbf-82dc-e92ae2fad011" path="/var/lib/kubelet/pods/09698a67-bfc2-4cbf-82dc-e92ae2fad011/volumes"
Sep 15 21:22:36.691884 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:36.691848    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69109814-6691-471e-bab0-1ba74fe84615" path="/var/lib/kubelet/pods/69109814-6691-471e-bab0-1ba74fe84615/volumes"
Sep 15 21:22:36.692173 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:36.692160    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" path="/var/lib/kubelet/pods/cf6df347-002d-4a10-aba7-5f5ac6b33b6f/volumes"
Sep 15 21:22:41.911431 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:41.911380    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:22:41.912005 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:41.911623    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" containerID="cri-o://138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506" gracePeriod=30
Sep 15 21:22:42.206072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.205989    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"]
Sep 15 21:22:42.206491 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206478    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" containerName="payload-processing"
Sep 15 21:22:42.206536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206494    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" containerName="payload-processing"
Sep 15 21:22:42.206536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206517    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69109814-6691-471e-bab0-1ba74fe84615" containerName="payload-pre-processing"
Sep 15 21:22:42.206536 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206523    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="69109814-6691-471e-bab0-1ba74fe84615" containerName="payload-pre-processing"
Sep 15 21:22:42.206674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206543    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09698a67-bfc2-4cbf-82dc-e92ae2fad011" containerName="revoke-keys"
Sep 15 21:22:42.206674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206549    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09698a67-bfc2-4cbf-82dc-e92ae2fad011" containerName="revoke-keys"
Sep 15 21:22:42.206674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206640    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="69109814-6691-471e-bab0-1ba74fe84615" containerName="payload-pre-processing"
Sep 15 21:22:42.206674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206652    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6df347-002d-4a10-aba7-5f5ac6b33b6f" containerName="payload-processing"
Sep 15 21:22:42.206674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.206661    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09698a67-bfc2-4cbf-82dc-e92ae2fad011" containerName="revoke-keys"
Sep 15 21:22:42.209475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.209449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:42.213074 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.213050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:22:42.213583 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.213548    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:22:42.221304 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.221281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"]
Sep 15 21:22:42.287331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.287297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-beb398-gc8zj\" (UID: \"1004c5f6-48df-45f6-a9fd-3a771179895c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:42.388265 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.388234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-beb398-gc8zj\" (UID: \"1004c5f6-48df-45f6-a9fd-3a771179895c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:42.388936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.388917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-beb398-gc8zj\" (UID: \"1004c5f6-48df-45f6-a9fd-3a771179895c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:42.521049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.521017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:42.819339 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:42.819312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"]
Sep 15 21:22:42.822387 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:22:42.822360    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1004c5f6_48df_45f6_a9fd_3a771179895c.slice/crio-62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a WatchSource:0}: Error finding container 62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a: Status 404 returned error can't find the container with id 62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a
Sep 15 21:22:43.165908 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:43.165878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj" event={"ID":"1004c5f6-48df-45f6-a9fd-3a771179895c","Type":"ContainerStarted","Data":"7d75deb934c21530095c754ce970500ef9a72f1c1cd4573f9dbfe1bf533c84f3"}
Sep 15 21:22:43.166311 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:43.165952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj" event={"ID":"1004c5f6-48df-45f6-a9fd-3a771179895c","Type":"ContainerStarted","Data":"62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a"}
Sep 15 21:22:43.185747 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:43.185699    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj" podStartSLOduration=1.185684772 podStartE2EDuration="1.185684772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:22:43.184370425 +0000 UTC m=+1470.970738433" watchObservedRunningTime="2026-09-15 21:22:43.185684772 +0000 UTC m=+1470.972052781"
Sep 15 21:22:44.174190 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:44.174155    2577 generic.go:356] "Generic (PLEG): container finished" podID="1004c5f6-48df-45f6-a9fd-3a771179895c" containerID="7d75deb934c21530095c754ce970500ef9a72f1c1cd4573f9dbfe1bf533c84f3" exitCode=0
Sep 15 21:22:44.174593 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:44.174243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj" event={"ID":"1004c5f6-48df-45f6-a9fd-3a771179895c","Type":"ContainerDied","Data":"7d75deb934c21530095c754ce970500ef9a72f1c1cd4573f9dbfe1bf533c84f3"}
Sep 15 21:22:45.334527 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:45.334502    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:45.414696 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:45.414667    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca\") pod \"1004c5f6-48df-45f6-a9fd-3a771179895c\" (UID: \"1004c5f6-48df-45f6-a9fd-3a771179895c\") "
Sep 15 21:22:45.415010 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:45.414989    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1004c5f6-48df-45f6-a9fd-3a771179895c" (UID: "1004c5f6-48df-45f6-a9fd-3a771179895c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:45.516452 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:45.516423    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1004c5f6-48df-45f6-a9fd-3a771179895c-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:46.187550 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:46.187520    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"
Sep 15 21:22:46.187788 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:46.187603    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj" event={"ID":"1004c5f6-48df-45f6-a9fd-3a771179895c","Type":"ContainerDied","Data":"62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a"}
Sep 15 21:22:46.187788 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:46.187659    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62de4c4a5555158fef04f994589e3a886d58b152287e7fab6058aa6183af647a"
Sep 15 21:22:46.624773 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:46.624737    2577 patch_prober.go:28] interesting pod/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 21:22:46.625158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:46.624796    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 21:22:47.168840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.168817    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:22:47.197237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.197153    2577 generic.go:356] "Generic (PLEG): container finished" podID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerID="138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506" exitCode=0
Sep 15 21:22:47.197237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.197232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerDied","Data":"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"}
Sep 15 21:22:47.197447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.197286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6" event={"ID":"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf","Type":"ContainerDied","Data":"ce9341d8a391cf74f5aca2619ec9f5a5a652092640933ae1ff51acd8ee0757d9"}
Sep 15 21:22:47.197447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.197309    2577 scope.go:117] "RemoveContainer" containerID="138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"
Sep 15 21:22:47.197447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.197288    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"
Sep 15 21:22:47.207267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.207250    2577 scope.go:117] "RemoveContainer" containerID="034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"
Sep 15 21:22:47.216182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.216159    2577 scope.go:117] "RemoveContainer" containerID="138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"
Sep 15 21:22:47.216468 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:22:47.216448    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506\": container with ID starting with 138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506 not found: ID does not exist" containerID="138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"
Sep 15 21:22:47.216552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.216480    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506"} err="failed to get container status \"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506\": rpc error: code = NotFound desc = could not find container \"138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506\": container with ID starting with 138dfc64cdcb99f112e7610aaf4fda08ab0078a1d91b9f7c63044a4fb6079506 not found: ID does not exist"
Sep 15 21:22:47.216552 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.216503    2577 scope.go:117] "RemoveContainer" containerID="034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"
Sep 15 21:22:47.216865 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:22:47.216846    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab\": container with ID starting with 034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab not found: ID does not exist" containerID="034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"
Sep 15 21:22:47.216928 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.216874    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab"} err="failed to get container status \"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab\": rpc error: code = NotFound desc = could not find container \"034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab\": container with ID starting with 034c85671965bd950b44253b49996eedf01e688b0a52742450f70ad99267a3ab not found: ID does not exist"
Sep 15 21:22:47.233387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233358    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4966\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233439    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233538 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233508    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233556    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.233804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233741    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket\") pod \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\" (UID: \"a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf\") "
Sep 15 21:22:47.234015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233854    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:22:47.234015 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.233948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:47.234126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234035    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:22:47.234126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:22:47.234353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234327    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:22:47.234482 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234461    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.234762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234489    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.234762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234504    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.234762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234518    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.234762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.234532    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.235993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.235962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966" (OuterVolumeSpecName: "kube-api-access-g4966") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "kube-api-access-g4966". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:47.236106 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.236017    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:47.236106 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.236055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:22:47.236259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.236240    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" (UID: "a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:22:47.336095 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.336062    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.336095 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.336093    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4966\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-kube-api-access-g4966\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.336095 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.336104    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.336368 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.336113    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:22:47.522740 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.522710    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:22:47.528599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:47.528529    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8f89be-openshift-default-cbc79b888-nflg6"]
Sep 15 21:22:48.691326 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:48.691290    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" path="/var/lib/kubelet/pods/a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf/volumes"
Sep 15 21:22:57.973492 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:57.973449    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"]
Sep 15 21:22:57.975518 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:57.975495    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-beb398-gc8zj"]
Sep 15 21:22:58.691393 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:22:58.691362    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1004c5f6-48df-45f6-a9fd-3a771179895c" path="/var/lib/kubelet/pods/1004c5f6-48df-45f6-a9fd-3a771179895c/volumes"
Sep 15 21:23:05.794712 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:05.794678    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:23:05.795107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:05.794885    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy" containerID="cri-o://cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429" gracePeriod=30
Sep 15 21:23:06.096139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096055    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"]
Sep 15 21:23:06.096511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096498    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:06.096603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096512    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:06.096603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096543    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1004c5f6-48df-45f6-a9fd-3a771179895c" containerName="revoke-keys"
Sep 15 21:23:06.096603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096550    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1004c5f6-48df-45f6-a9fd-3a771179895c" containerName="revoke-keys"
Sep 15 21:23:06.096720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096653    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:06.096720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.096667    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1004c5f6-48df-45f6-a9fd-3a771179895c" containerName="revoke-keys"
Sep 15 21:23:06.101340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.101322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:06.104197 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:06.104166    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" podUID="8109ae30-b204-4c95-b4de-4a74d6214edf" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:06.104363 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.104344    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:23:06.105786 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:06.105764    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-2.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-131-2.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 15 21:23:06.151960 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.151933    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"]
Sep 15 21:23:06.240739 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.240708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk\" (UID: \"8109ae30-b204-4c95-b4de-4a74d6214edf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:06.341497 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:06.341462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk\" (UID: \"8109ae30-b204-4c95-b4de-4a74d6214edf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:07.021486 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:07.021453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:23:07.022143 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:07.022123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk\" (UID: \"8109ae30-b204-4c95-b4de-4a74d6214edf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:07.311903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:07.311817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:07.590449 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:07.590413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"]
Sep 15 21:23:07.591039 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:23:07.591005    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8109ae30_b204_4c95_b4de_4a74d6214edf.slice/crio-1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8 WatchSource:0}: Error finding container 1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8: Status 404 returned error can't find the container with id 1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8
Sep 15 21:23:08.343021 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:08.342989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk" event={"ID":"8109ae30-b204-4c95-b4de-4a74d6214edf","Type":"ContainerStarted","Data":"dc04c135fff84cea03cc4f7ff96801eb8df99ca618f1a8fa03223662db0a03f8"}
Sep 15 21:23:08.343330 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:08.343091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk" event={"ID":"8109ae30-b204-4c95-b4de-4a74d6214edf","Type":"ContainerStarted","Data":"1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8"}
Sep 15 21:23:09.350964 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.350931    2577 generic.go:356] "Generic (PLEG): container finished" podID="8109ae30-b204-4c95-b4de-4a74d6214edf" containerID="dc04c135fff84cea03cc4f7ff96801eb8df99ca618f1a8fa03223662db0a03f8" exitCode=0
Sep 15 21:23:09.351357 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.351000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk" event={"ID":"8109ae30-b204-4c95-b4de-4a74d6214edf","Type":"ContainerDied","Data":"dc04c135fff84cea03cc4f7ff96801eb8df99ca618f1a8fa03223662db0a03f8"}
Sep 15 21:23:09.503583 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.503542    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:09.676372 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.676283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca\") pod \"8109ae30-b204-4c95-b4de-4a74d6214edf\" (UID: \"8109ae30-b204-4c95-b4de-4a74d6214edf\") "
Sep 15 21:23:09.676726 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.676702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8109ae30-b204-4c95-b4de-4a74d6214edf" (UID: "8109ae30-b204-4c95-b4de-4a74d6214edf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:09.777529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:09.777492    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8109ae30-b204-4c95-b4de-4a74d6214edf-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:10.362264 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:10.362228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk" event={"ID":"8109ae30-b204-4c95-b4de-4a74d6214edf","Type":"ContainerDied","Data":"1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8"}
Sep 15 21:23:10.362667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:10.362275    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e312faeacd71e12fb1f96e3ab3fa5a3d359c125b57ae53decc4faf5076789c8"
Sep 15 21:23:10.362667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:10.362243    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"
Sep 15 21:23:11.066664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.066609    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:23:11.193074 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193137    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193171    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193329    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmj6l\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193385    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:11.193465 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193446    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo\") pod \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\" (UID: \"38d8521d-6f8f-470a-9bc3-3bcd0a300953\") "
Sep 15 21:23:11.193761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:11.193761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193555    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data" (OuterVolumeSpecName: "istio-data") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:11.193761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:11.193761 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:11.193974 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193952    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.194039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193980    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.194039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.193994    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.194039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.194007    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.194039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.194021    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.195727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.195706    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:23:11.195888 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.195869    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l" (OuterVolumeSpecName: "kube-api-access-dmj6l") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "kube-api-access-dmj6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:11.196018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.196001    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:11.196062 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.196009    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token" (OuterVolumeSpecName: "istio-token") pod "38d8521d-6f8f-470a-9bc3-3bcd0a300953" (UID: "38d8521d-6f8f-470a-9bc3-3bcd0a300953"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:11.294862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.294829    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmj6l\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-kube-api-access-dmj6l\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.294862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.294855    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.294862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.294865    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.295088 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.294874    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d8521d-6f8f-470a-9bc3-3bcd0a300953-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:11.370076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.369985    2577 generic.go:356] "Generic (PLEG): container finished" podID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerID="cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429" exitCode=0
Sep 15 21:23:11.370076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.370039    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"
Sep 15 21:23:11.370506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.370038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerDied","Data":"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"}
Sep 15 21:23:11.370506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.370152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg" event={"ID":"38d8521d-6f8f-470a-9bc3-3bcd0a300953","Type":"ContainerDied","Data":"b65711c2997a95df27c876d69cf26eecda583499a2ec5819ebd832be908416fb"}
Sep 15 21:23:11.370506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.370177    2577 scope.go:117] "RemoveContainer" containerID="cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"
Sep 15 21:23:11.380197 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.380181    2577 scope.go:117] "RemoveContainer" containerID="1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"
Sep 15 21:23:11.388762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.388740    2577 scope.go:117] "RemoveContainer" containerID="cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"
Sep 15 21:23:11.389033 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:11.389015    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429\": container with ID starting with cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429 not found: ID does not exist" containerID="cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"
Sep 15 21:23:11.389092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.389043    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429"} err="failed to get container status \"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429\": rpc error: code = NotFound desc = could not find container \"cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429\": container with ID starting with cb3a47be103ff5da7b440f2414ec122bb243c7741608b067689ae4c8fef8c429 not found: ID does not exist"
Sep 15 21:23:11.389092 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.389061    2577 scope.go:117] "RemoveContainer" containerID="1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"
Sep 15 21:23:11.389319 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:11.389295    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23\": container with ID starting with 1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23 not found: ID does not exist" containerID="1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"
Sep 15 21:23:11.389362 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.389329    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23"} err="failed to get container status \"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23\": rpc error: code = NotFound desc = could not find container \"1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23\": container with ID starting with 1f471bb520dca41231814891d218f0f902b2d3f1c2dc16eb3e4f78e408023f23 not found: ID does not exist"
Sep 15 21:23:11.393125 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.393105    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:23:11.396737 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:11.396719    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-beb398-openshift-default-5fdf8c64f6-ht4cg"]
Sep 15 21:23:12.695998 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:12.695958    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" path="/var/lib/kubelet/pods/38d8521d-6f8f-470a-9bc3-3bcd0a300953/volumes"
Sep 15 21:23:12.747049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:12.747024    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:23:12.750377 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:12.750354    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:23:16.631632 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.631596    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:23:16.632146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.631835    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" podUID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" containerName="payload-processing" containerID="cri-o://0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1" gracePeriod=30
Sep 15 21:23:16.639621 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.639557    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:23:16.640021 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.639995    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" podUID="1f2878d0-2df8-4f72-8954-0612fb06695b" containerName="payload-pre-processing" containerID="cri-o://4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504" gracePeriod=30
Sep 15 21:23:16.795626 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.795591    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"]
Sep 15 21:23:16.797841 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.797814    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-7c59a9-5tsqk"]
Sep 15 21:23:16.924913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.924890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:23:16.927648 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.927629    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:23:16.951071 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951044    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume\") pod \"1f2878d0-2df8-4f72-8954-0612fb06695b\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") "
Sep 15 21:23:16.951071 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume\") pod \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") "
Sep 15 21:23:16.951300 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951112    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x676k\" (UniqueName: \"kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k\") pod \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\" (UID: \"7a1a4ef3-7927-4111-9485-9a893d24c6d0\") "
Sep 15 21:23:16.951300 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpkp8\" (UniqueName: \"kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8\") pod \"1f2878d0-2df8-4f72-8954-0612fb06695b\" (UID: \"1f2878d0-2df8-4f72-8954-0612fb06695b\") "
Sep 15 21:23:16.951420 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951348    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a1a4ef3-7927-4111-9485-9a893d24c6d0" (UID: "7a1a4ef3-7927-4111-9485-9a893d24c6d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:16.951541 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951432    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f2878d0-2df8-4f72-8954-0612fb06695b" (UID: "1f2878d0-2df8-4f72-8954-0612fb06695b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:16.951779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951761    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f2878d0-2df8-4f72-8954-0612fb06695b-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:16.951779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.951779    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a1a4ef3-7927-4111-9485-9a893d24c6d0-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:16.954007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.953973    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k" (OuterVolumeSpecName: "kube-api-access-x676k") pod "7a1a4ef3-7927-4111-9485-9a893d24c6d0" (UID: "7a1a4ef3-7927-4111-9485-9a893d24c6d0"). InnerVolumeSpecName "kube-api-access-x676k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:16.954111 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:16.954012    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8" (OuterVolumeSpecName: "kube-api-access-fpkp8") pod "1f2878d0-2df8-4f72-8954-0612fb06695b" (UID: "1f2878d0-2df8-4f72-8954-0612fb06695b"). InnerVolumeSpecName "kube-api-access-fpkp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:17.052909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.052877    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpkp8\" (UniqueName: \"kubernetes.io/projected/1f2878d0-2df8-4f72-8954-0612fb06695b-kube-api-access-fpkp8\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:17.052909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.052903    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x676k\" (UniqueName: \"kubernetes.io/projected/7a1a4ef3-7927-4111-9485-9a893d24c6d0-kube-api-access-x676k\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:17.411952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.411910    2577 generic.go:356] "Generic (PLEG): container finished" podID="1f2878d0-2df8-4f72-8954-0612fb06695b" containerID="4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504" exitCode=0
Sep 15 21:23:17.412163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.411982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"
Sep 15 21:23:17.412163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.412005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" event={"ID":"1f2878d0-2df8-4f72-8954-0612fb06695b","Type":"ContainerDied","Data":"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"}
Sep 15 21:23:17.412163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.412106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8" event={"ID":"1f2878d0-2df8-4f72-8954-0612fb06695b","Type":"ContainerDied","Data":"1d0a2b66bb1ab6714e88f8507a582990bffb0d28534f9ea2543b625bd040f34b"}
Sep 15 21:23:17.412163 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.412137    2577 scope.go:117] "RemoveContainer" containerID="4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"
Sep 15 21:23:17.413691 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.413667    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" containerID="0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1" exitCode=0
Sep 15 21:23:17.413828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.413738    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"
Sep 15 21:23:17.413828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.413764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" event={"ID":"7a1a4ef3-7927-4111-9485-9a893d24c6d0","Type":"ContainerDied","Data":"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"}
Sep 15 21:23:17.413828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.413818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s" event={"ID":"7a1a4ef3-7927-4111-9485-9a893d24c6d0","Type":"ContainerDied","Data":"bd092150433e6be8f740ff87b336293e7194941f92d28e816def26e7463b6b7a"}
Sep 15 21:23:17.422259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.422241    2577 scope.go:117] "RemoveContainer" containerID="4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"
Sep 15 21:23:17.422702 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:17.422521    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504\": container with ID starting with 4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504 not found: ID does not exist" containerID="4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"
Sep 15 21:23:17.422702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.422553    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504"} err="failed to get container status \"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504\": rpc error: code = NotFound desc = could not find container \"4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504\": container with ID starting with 4963db5d00a6f93e9d520820b4fe76ed7ab03e0641394122162c708639b30504 not found: ID does not exist"
Sep 15 21:23:17.422702 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.422630    2577 scope.go:117] "RemoveContainer" containerID="0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"
Sep 15 21:23:17.437212 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.437155    2577 scope.go:117] "RemoveContainer" containerID="0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"
Sep 15 21:23:17.437995 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:17.437968    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1\": container with ID starting with 0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1 not found: ID does not exist" containerID="0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"
Sep 15 21:23:17.438121 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.438005    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1"} err="failed to get container status \"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1\": rpc error: code = NotFound desc = could not find container \"0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1\": container with ID starting with 0a79e659705b119bfdb977fd63990d726ce1659ea5c4446bc75868b14b8e8fb1 not found: ID does not exist"
Sep 15 21:23:17.439655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.439628    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:23:17.442796 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.442774    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-7c59a9-59849c7845-l974s"]
Sep 15 21:23:17.452949 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.452931    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:23:17.454827 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:17.454807    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7c59a9-8545758794-r6wm8"]
Sep 15 21:23:18.691365 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:18.691332    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2878d0-2df8-4f72-8954-0612fb06695b" path="/var/lib/kubelet/pods/1f2878d0-2df8-4f72-8954-0612fb06695b/volumes"
Sep 15 21:23:18.691929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:18.691907    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" path="/var/lib/kubelet/pods/7a1a4ef3-7927-4111-9485-9a893d24c6d0/volumes"
Sep 15 21:23:18.692273 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:18.692255    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8109ae30-b204-4c95-b4de-4a74d6214edf" path="/var/lib/kubelet/pods/8109ae30-b204-4c95-b4de-4a74d6214edf/volumes"
Sep 15 21:23:23.344014 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.343972    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"]
Sep 15 21:23:23.344629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344609    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8109ae30-b204-4c95-b4de-4a74d6214edf" containerName="revoke-keys"
Sep 15 21:23:23.344629 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344630    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8109ae30-b204-4c95-b4de-4a74d6214edf" containerName="revoke-keys"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344644    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" containerName="payload-processing"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344653    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" containerName="payload-processing"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344661    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344668    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344687    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344696    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344732    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2878d0-2df8-4f72-8954-0612fb06695b" containerName="payload-pre-processing"
Sep 15 21:23:23.344800 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344740    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2878d0-2df8-4f72-8954-0612fb06695b" containerName="payload-pre-processing"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344846    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1a4ef3-7927-4111-9485-9a893d24c6d0" containerName="payload-processing"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344863    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37a4aaf-9cf0-4691-b23d-8fe47de2d1bf" containerName="istio-proxy"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344873    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2878d0-2df8-4f72-8954-0612fb06695b" containerName="payload-pre-processing"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344884    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344891    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:23.345194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.344901    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8109ae30-b204-4c95-b4de-4a74d6214edf" containerName="revoke-keys"
Sep 15 21:23:23.349593 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.349545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:23.353363 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.353101    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:23:23.353363 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.353121    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:23:23.354807 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.354783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"]
Sep 15 21:23:23.412512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.412480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9\" (UID: \"9da358ba-d4a8-4595-889d-bdf7673db420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:23.513913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.513878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9\" (UID: \"9da358ba-d4a8-4595-889d-bdf7673db420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:23.514473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.514453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9\" (UID: \"9da358ba-d4a8-4595-889d-bdf7673db420\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:23.661889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.661812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:23.935289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:23.935264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"]
Sep 15 21:23:24.464521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:24.464481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9" event={"ID":"9da358ba-d4a8-4595-889d-bdf7673db420","Type":"ContainerStarted","Data":"6c67c01962220d00f9c0ed1cc0cd1e0c98814fa2d089630c5d8fd3c6d3390f11"}
Sep 15 21:23:24.464910 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:24.464541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9" event={"ID":"9da358ba-d4a8-4595-889d-bdf7673db420","Type":"ContainerStarted","Data":"a19b8a1bd3d42c19b1d5fa89ce3ac65cf1bc73afc845ed5c34d4f89bd1a51c57"}
Sep 15 21:23:24.480133 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:24.480079    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9" podStartSLOduration=1.480063654 podStartE2EDuration="1.480063654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:23:24.478314062 +0000 UTC m=+1512.264682072" watchObservedRunningTime="2026-09-15 21:23:24.480063654 +0000 UTC m=+1512.266431662"
Sep 15 21:23:25.471670 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:25.471627    2577 generic.go:356] "Generic (PLEG): container finished" podID="9da358ba-d4a8-4595-889d-bdf7673db420" containerID="6c67c01962220d00f9c0ed1cc0cd1e0c98814fa2d089630c5d8fd3c6d3390f11" exitCode=0
Sep 15 21:23:25.472151 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:25.471829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9" event={"ID":"9da358ba-d4a8-4595-889d-bdf7673db420","Type":"ContainerDied","Data":"6c67c01962220d00f9c0ed1cc0cd1e0c98814fa2d089630c5d8fd3c6d3390f11"}
Sep 15 21:23:26.635591 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:26.635550    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:26.747375 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:26.747301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca\") pod \"9da358ba-d4a8-4595-889d-bdf7673db420\" (UID: \"9da358ba-d4a8-4595-889d-bdf7673db420\") "
Sep 15 21:23:26.747653 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:26.747632    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9da358ba-d4a8-4595-889d-bdf7673db420" (UID: "9da358ba-d4a8-4595-889d-bdf7673db420"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:26.848503 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:26.848471    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da358ba-d4a8-4595-889d-bdf7673db420-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:27.484014 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:27.483986    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"
Sep 15 21:23:27.484014 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:27.484001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9" event={"ID":"9da358ba-d4a8-4595-889d-bdf7673db420","Type":"ContainerDied","Data":"a19b8a1bd3d42c19b1d5fa89ce3ac65cf1bc73afc845ed5c34d4f89bd1a51c57"}
Sep 15 21:23:27.484221 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:27.484047    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a19b8a1bd3d42c19b1d5fa89ce3ac65cf1bc73afc845ed5c34d4f89bd1a51c57"
Sep 15 21:23:33.889454 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:33.889416    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:23:33.889999 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:33.889714    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" podUID="c84a630a-2886-425f-b279-3cbed2ac3072" containerName="payload-processing" containerID="cri-o://e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41" gracePeriod=30
Sep 15 21:23:33.895252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:33.894618    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:23:33.895252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:33.894828    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" podUID="2fe59df2-77f5-49b0-92b0-b12124af31c7" containerName="payload-pre-processing" containerID="cri-o://bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df" gracePeriod=30
Sep 15 21:23:34.058938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.058907    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"]
Sep 15 21:23:34.062496 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.062284    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2c7774-rh9k9"]
Sep 15 21:23:34.163694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.163664    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:23:34.194640 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.194611    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:23:34.224342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.224310    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume\") pod \"c84a630a-2886-425f-b279-3cbed2ac3072\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") "
Sep 15 21:23:34.224540 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.224464    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76g7l\" (UniqueName: \"kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l\") pod \"c84a630a-2886-425f-b279-3cbed2ac3072\" (UID: \"c84a630a-2886-425f-b279-3cbed2ac3072\") "
Sep 15 21:23:34.224858 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.224819    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c84a630a-2886-425f-b279-3cbed2ac3072" (UID: "c84a630a-2886-425f-b279-3cbed2ac3072"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:34.237414 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.235352    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l" (OuterVolumeSpecName: "kube-api-access-76g7l") pod "c84a630a-2886-425f-b279-3cbed2ac3072" (UID: "c84a630a-2886-425f-b279-3cbed2ac3072"). InnerVolumeSpecName "kube-api-access-76g7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:34.325326 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.325297    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spc88\" (UniqueName: \"kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88\") pod \"2fe59df2-77f5-49b0-92b0-b12124af31c7\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") "
Sep 15 21:23:34.325514 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.325357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume\") pod \"2fe59df2-77f5-49b0-92b0-b12124af31c7\" (UID: \"2fe59df2-77f5-49b0-92b0-b12124af31c7\") "
Sep 15 21:23:34.325634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.325617    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c84a630a-2886-425f-b279-3cbed2ac3072-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:34.325708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.325639    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76g7l\" (UniqueName: \"kubernetes.io/projected/c84a630a-2886-425f-b279-3cbed2ac3072-kube-api-access-76g7l\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:34.325805 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.325782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fe59df2-77f5-49b0-92b0-b12124af31c7" (UID: "2fe59df2-77f5-49b0-92b0-b12124af31c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:34.327395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.327374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88" (OuterVolumeSpecName: "kube-api-access-spc88") pod "2fe59df2-77f5-49b0-92b0-b12124af31c7" (UID: "2fe59df2-77f5-49b0-92b0-b12124af31c7"). InnerVolumeSpecName "kube-api-access-spc88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:34.426685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.426592    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spc88\" (UniqueName: \"kubernetes.io/projected/2fe59df2-77f5-49b0-92b0-b12124af31c7-kube-api-access-spc88\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:34.426685 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.426625    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fe59df2-77f5-49b0-92b0-b12124af31c7-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:34.524558 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.524525    2577 generic.go:356] "Generic (PLEG): container finished" podID="c84a630a-2886-425f-b279-3cbed2ac3072" containerID="e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41" exitCode=0
Sep 15 21:23:34.524801 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.524612    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"
Sep 15 21:23:34.524801 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.524610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" event={"ID":"c84a630a-2886-425f-b279-3cbed2ac3072","Type":"ContainerDied","Data":"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"}
Sep 15 21:23:34.524801 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.524767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg" event={"ID":"c84a630a-2886-425f-b279-3cbed2ac3072","Type":"ContainerDied","Data":"5bfb9aa6aa24ce94b529f12070e3e97d27bf499be431dae1e066eb543febee82"}
Sep 15 21:23:34.524978 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.524829    2577 scope.go:117] "RemoveContainer" containerID="e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"
Sep 15 21:23:34.526283 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.526251    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fe59df2-77f5-49b0-92b0-b12124af31c7" containerID="bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df" exitCode=0
Sep 15 21:23:34.526393 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.526279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" event={"ID":"2fe59df2-77f5-49b0-92b0-b12124af31c7","Type":"ContainerDied","Data":"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"}
Sep 15 21:23:34.526393 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.526306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"
Sep 15 21:23:34.526393 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.526321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss" event={"ID":"2fe59df2-77f5-49b0-92b0-b12124af31c7","Type":"ContainerDied","Data":"6e29c5060b74ca12b7a862f21d734c25a8290bca2f7c2893f87045b56dafc0bb"}
Sep 15 21:23:34.535295 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.535274    2577 scope.go:117] "RemoveContainer" containerID="e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"
Sep 15 21:23:34.535650 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:34.535623    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41\": container with ID starting with e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41 not found: ID does not exist" containerID="e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"
Sep 15 21:23:34.535722 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.535661    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41"} err="failed to get container status \"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41\": rpc error: code = NotFound desc = could not find container \"e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41\": container with ID starting with e40250812c463c9e1d71b3b24a2ba5123ea8168f4dfea602c02b92ecb282fe41 not found: ID does not exist"
Sep 15 21:23:34.535722 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.535682    2577 scope.go:117] "RemoveContainer" containerID="bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"
Sep 15 21:23:34.544394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.544373    2577 scope.go:117] "RemoveContainer" containerID="bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"
Sep 15 21:23:34.544679 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:34.544659    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df\": container with ID starting with bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df not found: ID does not exist" containerID="bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"
Sep 15 21:23:34.544755 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.544686    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df"} err="failed to get container status \"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df\": rpc error: code = NotFound desc = could not find container \"bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df\": container with ID starting with bf032112c5fa1df68dd00709ffb265e20f3df7b1a21668f114d875c44a6903df not found: ID does not exist"
Sep 15 21:23:34.560129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.560104    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:23:34.564173 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.564155    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-2c7774-8467959cc4-xxwss"]
Sep 15 21:23:34.581342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.581313    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:23:34.588111 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.588085    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2c7774-6f95c566f5-pmzrg"]
Sep 15 21:23:34.690353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.690275    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe59df2-77f5-49b0-92b0-b12124af31c7" path="/var/lib/kubelet/pods/2fe59df2-77f5-49b0-92b0-b12124af31c7/volumes"
Sep 15 21:23:34.690632 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.690613    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9da358ba-d4a8-4595-889d-bdf7673db420" path="/var/lib/kubelet/pods/9da358ba-d4a8-4595-889d-bdf7673db420/volumes"
Sep 15 21:23:34.690931 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:34.690920    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84a630a-2886-425f-b279-3cbed2ac3072" path="/var/lib/kubelet/pods/c84a630a-2886-425f-b279-3cbed2ac3072/volumes"
Sep 15 21:23:40.354019 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:40.353987    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:23:40.354419 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:40.354239    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy" containerID="cri-o://5f986c96be151db56db6d9e877a7a39d8c4c28c04b9be4e5807c5b973c8e6174" gracePeriod=30
Sep 15 21:23:45.598920 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.598890    2577 generic.go:356] "Generic (PLEG): container finished" podID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerID="5f986c96be151db56db6d9e877a7a39d8c4c28c04b9be4e5807c5b973c8e6174" exitCode=0
Sep 15 21:23:45.599250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.598964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerDied","Data":"5f986c96be151db56db6d9e877a7a39d8c4c28c04b9be4e5807c5b973c8e6174"}
Sep 15 21:23:45.599250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.599022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4" event={"ID":"fcf514ad-b83d-4498-ad21-c4c093a5212b","Type":"ContainerDied","Data":"2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d"}
Sep 15 21:23:45.599250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.599039    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e8271bd05e2b1f225440f7d3c891593313e57979103313b5a5bb27a05e9ff0d"
Sep 15 21:23:45.599250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.599053    2577 scope.go:117] "RemoveContainer" containerID="b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b"
Sep 15 21:23:45.610289 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:23:45.610261    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b\": container with ID starting with b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b not found: ID does not exist" containerID="b9e72de8df2fd4c2e1d5b81604530880713572d10d73036cc26d86af5a77da8b"
Sep 15 21:23:45.613932 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.613907    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:23:45.742642 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.742844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742668    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.742925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742857    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.742925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743035 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742944    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743035 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742970    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:45.743035 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.742980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743035 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743005    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:23:45.743246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97hc5\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743246 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743175    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket\") pod \"fcf514ad-b83d-4498-ad21-c4c093a5212b\" (UID: \"fcf514ad-b83d-4498-ad21-c4c093a5212b\") "
Sep 15 21:23:45.743434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743107    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:45.743434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743352    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:45.743434 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:45.743745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743652    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.743745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743674    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.743745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743687    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf514ad-b83d-4498-ad21-c4c093a5212b-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.743745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743699    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.743745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.743710    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.745391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.745368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:23:45.745664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.745444    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:23:45.745664 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.745496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5" (OuterVolumeSpecName: "kube-api-access-97hc5") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "kube-api-access-97hc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:45.745795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.745781    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcf514ad-b83d-4498-ad21-c4c093a5212b" (UID: "fcf514ad-b83d-4498-ad21-c4c093a5212b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:23:45.844806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.844774    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97hc5\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-kube-api-access-97hc5\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.844806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.844802    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.844806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.844811    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:45.845031 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:45.844818    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf514ad-b83d-4498-ad21-c4c093a5212b-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:23:46.606323 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:46.606298    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"
Sep 15 21:23:46.631950 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:46.631919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:23:46.634356 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:46.634330    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2c7774-openshift-default-7bdb86c5d4-jrpm4"]
Sep 15 21:23:46.690900 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:46.690868    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" path="/var/lib/kubelet/pods/fcf514ad-b83d-4498-ad21-c4c093a5212b/volumes"
Sep 15 21:23:51.296174 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296144    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:23:51.296634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296616    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296637    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296647    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9da358ba-d4a8-4595-889d-bdf7673db420" containerName="revoke-keys"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296653    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9da358ba-d4a8-4595-889d-bdf7673db420" containerName="revoke-keys"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296665    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe59df2-77f5-49b0-92b0-b12124af31c7" containerName="payload-pre-processing"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296672    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe59df2-77f5-49b0-92b0-b12124af31c7" containerName="payload-pre-processing"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296683    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:51.296692 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296687    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d8521d-6f8f-470a-9bc3-3bcd0a300953" containerName="istio-proxy"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296697    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296706    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296716    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84a630a-2886-425f-b279-3cbed2ac3072" containerName="payload-processing"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296721    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84a630a-2886-425f-b279-3cbed2ac3072" containerName="payload-processing"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296816    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9da358ba-d4a8-4595-889d-bdf7673db420" containerName="revoke-keys"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296825    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c84a630a-2886-425f-b279-3cbed2ac3072" containerName="payload-processing"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296833    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296839    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe59df2-77f5-49b0-92b0-b12124af31c7" containerName="payload-pre-processing"
Sep 15 21:23:51.296926 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.296846    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf514ad-b83d-4498-ad21-c4c093a5212b" containerName="istio-proxy"
Sep 15 21:23:51.301200 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.301182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.304897 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.304871    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"kube-root-ca.crt\""
Sep 15 21:23:51.305039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.304878    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"sub-test-model-41ffdf-kserve-self-signed-certs\""
Sep 15 21:23:51.305039 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.304881    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"default-dockercfg-2p5wv\""
Sep 15 21:23:51.305175 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.304902    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"openshift-service-ca.crt\""
Sep 15 21:23:51.307813 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.307787    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:23:51.394511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.394511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.394749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.394749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.394749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.394749 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.394693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2hgm\" (UniqueName: \"kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495474 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495474 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495729 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2hgm\" (UniqueName: \"kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.495952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.496052 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.495995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.498068 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.498045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.498228 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.498209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.509040 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.509014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2hgm\" (UniqueName: \"kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm\") pod \"sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.612551 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.612457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:23:51.887969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:51.887943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:23:51.889902 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:23:51.889877    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d2bf6af_9faf_4fa9_887d_e37df9b71ab6.slice/crio-99a4d4c94435ec451ac80c09b6259c44b5fa1f69afe8ef0b6d8d4a8964b9519e WatchSource:0}: Error finding container 99a4d4c94435ec451ac80c09b6259c44b5fa1f69afe8ef0b6d8d4a8964b9519e: Status 404 returned error can't find the container with id 99a4d4c94435ec451ac80c09b6259c44b5fa1f69afe8ef0b6d8d4a8964b9519e
Sep 15 21:23:52.643828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:52.643792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" event={"ID":"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6","Type":"ContainerStarted","Data":"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"}
Sep 15 21:23:52.644267 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:52.643839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" event={"ID":"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6","Type":"ContainerStarted","Data":"99a4d4c94435ec451ac80c09b6259c44b5fa1f69afe8ef0b6d8d4a8964b9519e"}
Sep 15 21:23:52.662373 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:23:52.662318    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" podStartSLOduration=1.662302951 podStartE2EDuration="1.662302951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:23:52.660182676 +0000 UTC m=+1540.446550696" watchObservedRunningTime="2026-09-15 21:23:52.662302951 +0000 UTC m=+1540.448670974"
Sep 15 21:24:01.612727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:01.612695    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:24:01.613147 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:01.612740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:24:01.625071 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:01.625047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:24:01.705930 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:01.705903    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:24:13.112617 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:13.112584    2577 scope.go:117] "RemoveContainer" containerID="fc2007002b8e9b59955f90cfaab0e5de8dac06da79df97d098cc65e10a2c9bc4"
Sep 15 21:24:13.121284 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:13.121263    2577 scope.go:117] "RemoveContainer" containerID="bcf554bfabb408a71ab9b2cc762615adb9a1ad4a9608c462f0858b6f3a38389d"
Sep 15 21:24:13.129127 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:13.129107    2577 scope.go:117] "RemoveContainer" containerID="5f986c96be151db56db6d9e877a7a39d8c4c28c04b9be4e5807c5b973c8e6174"
Sep 15 21:24:50.524364 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:24:50.524333    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:25:05.545478 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.545394    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" podUID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" containerName="main" containerID="cri-o://14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd" gracePeriod=45
Sep 15 21:25:05.797822 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.797763    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:25:05.824169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824205    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824333 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824314    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824374    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2hgm\" (UniqueName: \"kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm\") pod \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\" (UID: \"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6\") "
Sep 15 21:25:05.824599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:05.824599 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:05.824829 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home" (OuterVolumeSpecName: "home") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:05.824923 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824826    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-model-cache\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.824923 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.824846    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tmp-dir\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.826831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.826805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm" (OuterVolumeSpecName: "kube-api-access-j2hgm") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "kube-api-access-j2hgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:25:05.826976 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.826954    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm" (OuterVolumeSpecName: "dshm") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:05.827116 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.827091    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" (UID: "9d2bf6af-9faf-4fa9-887d-e37df9b71ab6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:25:05.926193 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.926160    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-dshm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.926193 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.926188    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-tls-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.926193 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.926197    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-home\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.926417 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:05.926205    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2hgm\" (UniqueName: \"kubernetes.io/projected/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6-kube-api-access-j2hgm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:25:06.062866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.062779    2577 generic.go:356] "Generic (PLEG): container finished" podID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" containerID="14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd" exitCode=0
Sep 15 21:25:06.062866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.062831    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"
Sep 15 21:25:06.062866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.062843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" event={"ID":"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6","Type":"ContainerDied","Data":"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"}
Sep 15 21:25:06.063099 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.062882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c" event={"ID":"9d2bf6af-9faf-4fa9-887d-e37df9b71ab6","Type":"ContainerDied","Data":"99a4d4c94435ec451ac80c09b6259c44b5fa1f69afe8ef0b6d8d4a8964b9519e"}
Sep 15 21:25:06.063099 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.062901    2577 scope.go:117] "RemoveContainer" containerID="14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"
Sep 15 21:25:06.079602 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.079578    2577 scope.go:117] "RemoveContainer" containerID="14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"
Sep 15 21:25:06.079905 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:25:06.079886    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd\": container with ID starting with 14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd not found: ID does not exist" containerID="14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"
Sep 15 21:25:06.079960 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.079914    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd"} err="failed to get container status \"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd\": rpc error: code = NotFound desc = could not find container \"14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd\": container with ID starting with 14016fc58012251c79a38c3deecb73b0756268afd51bdddef0c06e7b038544fd not found: ID does not exist"
Sep 15 21:25:06.113140 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.113107    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:25:06.119956 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.119918    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/sub-test-model-41ffdf-kserve-78f58cc98f-xjl2c"]
Sep 15 21:25:06.690667 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:06.690632    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" path="/var/lib/kubelet/pods/9d2bf6af-9faf-4fa9-887d-e37df9b71ab6/volumes"
Sep 15 21:25:13.171385 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:13.171351    2577 scope.go:117] "RemoveContainer" containerID="afe131c3e09fb61cb6f0aff87f15ea49eb796f6be2091f1b021bd1d071612d32"
Sep 15 21:25:30.698179 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.698145    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:25:30.698969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.698943    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" containerName="main"
Sep 15 21:25:30.698969 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.698971    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" containerName="main"
Sep 15 21:25:30.699137 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.699079    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2bf6af-9faf-4fa9-887d-e37df9b71ab6" containerName="main"
Sep 15 21:25:30.703616 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.703594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.707528 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.707502    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"rate-test-model-41ffdf-kserve-self-signed-certs\""
Sep 15 21:25:30.708747 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.708728    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"openshift-service-ca.crt\""
Sep 15 21:25:30.708882 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.708765    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"kube-root-ca.crt\""
Sep 15 21:25:30.708882 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.708735    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-41ffdf\"/\"default-dockercfg-2p5wv\""
Sep 15 21:25:30.720625 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.720520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:25:30.856198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.856198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.856522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.856522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.856522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxrb\" (UniqueName: \"kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.856522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.856393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957658 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957658 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957658 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.957948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxrb\" (UniqueName: \"kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.958106 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.957958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.958106 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.958006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.958197 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.958130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.960127 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.960103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.960266 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.960251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:30.967112 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:30.967076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxrb\" (UniqueName: \"kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb\") pod \"rate-test-model-41ffdf-kserve-755b64d58c-p6tt8\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") " pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:31.014305 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:31.014276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:31.292200 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:31.292169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:25:31.294558 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:25:31.294534    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20fea755_3381_4840_8766_460b9aa2320e.slice/crio-ca1d0cb9d9fc78cfebcaea43adfca10be4a9c6b287d69d2325519b92d5b85f76 WatchSource:0}: Error finding container ca1d0cb9d9fc78cfebcaea43adfca10be4a9c6b287d69d2325519b92d5b85f76: Status 404 returned error can't find the container with id ca1d0cb9d9fc78cfebcaea43adfca10be4a9c6b287d69d2325519b92d5b85f76
Sep 15 21:25:32.211909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:32.211873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" event={"ID":"20fea755-3381-4840-8766-460b9aa2320e","Type":"ContainerStarted","Data":"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"}
Sep 15 21:25:32.211909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:32.211915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" event={"ID":"20fea755-3381-4840-8766-460b9aa2320e","Type":"ContainerStarted","Data":"ca1d0cb9d9fc78cfebcaea43adfca10be4a9c6b287d69d2325519b92d5b85f76"}
Sep 15 21:25:32.236859 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:32.236801    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" podStartSLOduration=2.2367802 podStartE2EDuration="2.2367802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:25:32.232099674 +0000 UTC m=+1640.018467683" watchObservedRunningTime="2026-09-15 21:25:32.2367802 +0000 UTC m=+1640.023148210"
Sep 15 21:25:41.014739 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:41.014691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:41.014739 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:41.014757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:41.027260 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:41.027235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:25:41.273252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:25:41.273171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:26:07.626089 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:07.626055    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:26:08.429053 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.428996    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:26:08.435167 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.435132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.441441 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.441412    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-b8f396-openshift-default-dockercfg-scwqc\""
Sep 15 21:26:08.445329 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.445305    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:26:08.536608 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.536815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.536815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.536815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.536815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25kq9\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.537061 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.536881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.537061 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.537000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.537061 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.537039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.537237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.537069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638166 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638166 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25kq9\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.638742 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.638738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.639130 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.639034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.639130 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.639097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.639228 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.639204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.639444 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.639423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.640679 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.640658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.640986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.640960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.649022 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.649000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.649252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.649223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25kq9\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9\") pod \"e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") " pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:08.748686 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:08.748649    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:09.034997 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.034961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:26:09.036725 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:26:09.036686    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dde8f1_ae31_4a24_b219_21efcd5ed514.slice/crio-64ec92581ffcb6ce16f35372170393bf17206906adbe7e21cd442ea488c5db96 WatchSource:0}: Error finding container 64ec92581ffcb6ce16f35372170393bf17206906adbe7e21cd442ea488c5db96: Status 404 returned error can't find the container with id 64ec92581ffcb6ce16f35372170393bf17206906adbe7e21cd442ea488c5db96
Sep 15 21:26:09.039073 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.039038    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:09.039158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.039116    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:09.039158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.039145    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:09.417038 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.416942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerStarted","Data":"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"}
Sep 15 21:26:09.417038 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.417000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerStarted","Data":"64ec92581ffcb6ce16f35372170393bf17206906adbe7e21cd442ea488c5db96"}
Sep 15 21:26:09.442603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.442533    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podStartSLOduration=1.442517254 podStartE2EDuration="1.442517254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:09.442445012 +0000 UTC m=+1677.228813020" watchObservedRunningTime="2026-09-15 21:26:09.442517254 +0000 UTC m=+1677.228885240"
Sep 15 21:26:09.748789 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:09.748754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:10.754752 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:10.754722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:11.428335 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:11.428308    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:11.429458 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:11.429433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:13.939544 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:13.939508    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:26:13.943469 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:13.943447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:13.946440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:13.946405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b8f396\""
Sep 15 21:26:13.955750 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:13.955724    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:26:14.093750 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.093708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8prcf\" (UniqueName: \"kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.093948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.093789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.194806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.194714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.194993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.194830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8prcf\" (UniqueName: \"kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.195473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.195449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.203674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.203645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8prcf\" (UniqueName: \"kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf\") pod \"payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.254484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.254461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:14.529199 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:14.529171    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:26:14.531470 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:26:14.531438    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9204c6_087d_4eb1_ad2b_abce7cdea605.slice/crio-db7a447d9e2eaca1096b9f26d8aca3b2ae5c464ccf5bf71b4b795dc6a678ad71 WatchSource:0}: Error finding container db7a447d9e2eaca1096b9f26d8aca3b2ae5c464ccf5bf71b4b795dc6a678ad71: Status 404 returned error can't find the container with id db7a447d9e2eaca1096b9f26d8aca3b2ae5c464ccf5bf71b4b795dc6a678ad71
Sep 15 21:26:15.452534 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:15.452494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" event={"ID":"fd9204c6-087d-4eb1-ad2b-abce7cdea605","Type":"ContainerStarted","Data":"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"}
Sep 15 21:26:15.452936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:15.452559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" event={"ID":"fd9204c6-087d-4eb1-ad2b-abce7cdea605","Type":"ContainerStarted","Data":"db7a447d9e2eaca1096b9f26d8aca3b2ae5c464ccf5bf71b4b795dc6a678ad71"}
Sep 15 21:26:15.452936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:15.452618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:15.468441 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:15.468394    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" podStartSLOduration=2.4683807 podStartE2EDuration="2.4683807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:15.467237371 +0000 UTC m=+1683.253605402" watchObservedRunningTime="2026-09-15 21:26:15.4683807 +0000 UTC m=+1683.254748708"
Sep 15 21:26:22.655336 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.655272    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" podUID="20fea755-3381-4840-8766-460b9aa2320e" containerName="main" containerID="cri-o://7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed" gracePeriod=45
Sep 15 21:26:22.917778 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.917723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:26:22.996856 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.996823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.996865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gxrb\" (UniqueName: \"kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.996894    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.996910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.996941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir\") pod \"20fea755-3381-4840-8766-460b9aa2320e\" (UID: \"20fea755-3381-4840-8766-460b9aa2320e\") "
Sep 15 21:26:22.997329 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997211    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache" (OuterVolumeSpecName: "model-cache") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:26:22.997329 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home" (OuterVolumeSpecName: "home") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:26:22.997415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997368    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-model-cache\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:22.997415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997388    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-home\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:22.997415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.997400    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:26:22.999103 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.999082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb" (OuterVolumeSpecName: "kube-api-access-6gxrb") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "kube-api-access-6gxrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:26:22.999528 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.999513    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:26:22.999583 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:22.999529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm" (OuterVolumeSpecName: "dshm") pod "20fea755-3381-4840-8766-460b9aa2320e" (UID: "20fea755-3381-4840-8766-460b9aa2320e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:26:23.098727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.098689    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-tmp-dir\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:23.098727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.098719    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fea755-3381-4840-8766-460b9aa2320e-dshm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:23.098727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.098733    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gxrb\" (UniqueName: \"kubernetes.io/projected/20fea755-3381-4840-8766-460b9aa2320e-kube-api-access-6gxrb\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:23.098979 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.098747    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fea755-3381-4840-8766-460b9aa2320e-tls-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:26:23.503864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.503830    2577 generic.go:356] "Generic (PLEG): container finished" podID="20fea755-3381-4840-8766-460b9aa2320e" containerID="7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed" exitCode=0
Sep 15 21:26:23.504049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.503894    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"
Sep 15 21:26:23.504049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.503930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" event={"ID":"20fea755-3381-4840-8766-460b9aa2320e","Type":"ContainerDied","Data":"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"}
Sep 15 21:26:23.504049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.503976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8" event={"ID":"20fea755-3381-4840-8766-460b9aa2320e","Type":"ContainerDied","Data":"ca1d0cb9d9fc78cfebcaea43adfca10be4a9c6b287d69d2325519b92d5b85f76"}
Sep 15 21:26:23.504049 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.503995    2577 scope.go:117] "RemoveContainer" containerID="7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"
Sep 15 21:26:23.513512 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.513496    2577 scope.go:117] "RemoveContainer" containerID="7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"
Sep 15 21:26:23.513794 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:26:23.513776    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed\": container with ID starting with 7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed not found: ID does not exist" containerID="7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"
Sep 15 21:26:23.513862 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.513801    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed"} err="failed to get container status \"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed\": rpc error: code = NotFound desc = could not find container \"7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed\": container with ID starting with 7fdd039c01c81ce3a91bad5a1e97cb2bd908c0530467e91bc7b3f541882e70ed not found: ID does not exist"
Sep 15 21:26:23.525608 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.525585    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:26:23.529557 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:23.529536    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-41ffdf/rate-test-model-41ffdf-kserve-755b64d58c-p6tt8"]
Sep 15 21:26:24.692450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:24.692410    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fea755-3381-4840-8766-460b9aa2320e" path="/var/lib/kubelet/pods/20fea755-3381-4840-8766-460b9aa2320e/volumes"
Sep 15 21:26:26.460857 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:26.460828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:26:26.723113 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:26.722244    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": read tcp 10.134.0.2:39902->10.134.0.97:15021: read: connection reset by peer" start-of-body=
Sep 15 21:26:26.723113 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:26.722310    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": read tcp 10.134.0.2:39902->10.134.0.97:15021: read: connection reset by peer"
Sep 15 21:26:27.534468 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.534431    2577 generic.go:356] "Generic (PLEG): container finished" podID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerID="eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63" exitCode=0
Sep 15 21:26:27.534886 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.534505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerDied","Data":"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"}
Sep 15 21:26:27.534970 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.534954    2577 scope.go:117] "RemoveContainer" containerID="eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"
Sep 15 21:26:27.536247 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.536216    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:27.536349 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.536285    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:27.536349 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:27.536314    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 15 21:26:28.550237 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:28.550197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerStarted","Data":"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"}
Sep 15 21:26:28.749578 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:28.749519    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:28.749794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:28.749669    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 15 21:26:28.749794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:28.749714    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 15 21:26:29.749368 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:29.749331    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 15 21:26:29.749971 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:29.749938    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 15 21:26:30.749584 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:30.749524    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 15 21:26:30.750183 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:30.750147    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 15 21:26:31.750181 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:31.750130    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 15 21:26:31.750708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:31.750218    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 15 21:26:32.749877 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:32.749838    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Sep 15 21:26:32.750087 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:32.749911    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Sep 15 21:26:33.752734 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:33.752700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:33.753198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:33.752983    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:33.753769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:33.753750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:26:44.436010 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.435972    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:26:44.436506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.436488    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fea755-3381-4840-8766-460b9aa2320e" containerName="main"
Sep 15 21:26:44.436556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.436509    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fea755-3381-4840-8766-460b9aa2320e" containerName="main"
Sep 15 21:26:44.436642 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.436607    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fea755-3381-4840-8766-460b9aa2320e" containerName="main"
Sep 15 21:26:44.439605 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.439589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.442357 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.442330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-ffb334\""
Sep 15 21:26:44.442513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.442420    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-ffb334-dockercfg-zh99l\""
Sep 15 21:26:44.451354 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.451331    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:26:44.461869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.461840    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:26:44.465677 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.465651    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.480577 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.480528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:26:44.508511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.508478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.508511 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.508518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn2d4\" (UniqueName: \"kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.609408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.609373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.609408 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.609411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn2d4\" (UniqueName: \"kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.609661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.609545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svzgz\" (UniqueName: \"kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.609661 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.609624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.610101 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.610082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.618364 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.618333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn2d4\" (UniqueName: \"kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4\") pod \"payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.710694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.710612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svzgz\" (UniqueName: \"kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.710694 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.710658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.711355 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.711335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.719828 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.719808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svzgz\" (UniqueName: \"kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz\") pod \"payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:44.752352 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.752319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:44.786865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:44.786832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:45.028282 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.028213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:26:45.030890 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:26:45.030857    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c6ca6a_08cb_4823_abce_c570f46c5886.slice/crio-98e59dc14372e8a6ad3f7b087a126ecd0dff34ed36ee6ceccaac380928c44d7d WatchSource:0}: Error finding container 98e59dc14372e8a6ad3f7b087a126ecd0dff34ed36ee6ceccaac380928c44d7d: Status 404 returned error can't find the container with id 98e59dc14372e8a6ad3f7b087a126ecd0dff34ed36ee6ceccaac380928c44d7d
Sep 15 21:26:45.074096 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.074068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:26:45.075364 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:26:45.075339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f5b470_594e_4c1e_98c6_ff0daee88ff4.slice/crio-144d99bc779cdfad3c7a81750dfc1617bb89790ef9b30a4484b58864b1623505 WatchSource:0}: Error finding container 144d99bc779cdfad3c7a81750dfc1617bb89790ef9b30a4484b58864b1623505: Status 404 returned error can't find the container with id 144d99bc779cdfad3c7a81750dfc1617bb89790ef9b30a4484b58864b1623505
Sep 15 21:26:45.655472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.655440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" event={"ID":"15c6ca6a-08cb-4823-abce-c570f46c5886","Type":"ContainerStarted","Data":"de827feb10f4df0fca0a5ea9371f9cab802935f0b2bf8eed83cd85e767790b01"}
Sep 15 21:26:45.655983 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.655503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" event={"ID":"15c6ca6a-08cb-4823-abce-c570f46c5886","Type":"ContainerStarted","Data":"98e59dc14372e8a6ad3f7b087a126ecd0dff34ed36ee6ceccaac380928c44d7d"}
Sep 15 21:26:45.655983 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.655537    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:26:45.657418 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.657387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" event={"ID":"20f5b470-594e-4c1e-98c6-ff0daee88ff4","Type":"ContainerStarted","Data":"b3984f13e2b711a78e9dde9ead47ee2b01afdee8d9d2a8dc5301e8c1f7b392ad"}
Sep 15 21:26:45.657556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.657436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" event={"ID":"20f5b470-594e-4c1e-98c6-ff0daee88ff4","Type":"ContainerStarted","Data":"144d99bc779cdfad3c7a81750dfc1617bb89790ef9b30a4484b58864b1623505"}
Sep 15 21:26:45.657556 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.657467    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:45.675719 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.675660    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" podStartSLOduration=1.675644256 podStartE2EDuration="1.675644256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:45.673767604 +0000 UTC m=+1713.460135607" watchObservedRunningTime="2026-09-15 21:26:45.675644256 +0000 UTC m=+1713.462012330"
Sep 15 21:26:45.698032 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:45.697982    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" podStartSLOduration=1.697963851 podStartE2EDuration="1.697963851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:45.697195586 +0000 UTC m=+1713.483563596" watchObservedRunningTime="2026-09-15 21:26:45.697963851 +0000 UTC m=+1713.484331861"
Sep 15 21:26:56.665762 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:56.665724    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:26:56.666334 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:26:56.665957    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:27:15.462785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:15.462751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"
Sep 15 21:27:15.575241 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:15.575207    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w_20f5b470-594e-4c1e-98c6-ff0daee88ff4/payload-processing/0.log"
Sep 15 21:27:46.319122 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:46.319090    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"
Sep 15 21:27:46.431720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:46.431690    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w_20f5b470-594e-4c1e-98c6-ff0daee88ff4/payload-processing/0.log"
Sep 15 21:27:53.897659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.897620    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:27:53.900924 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.900907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:53.903524 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.903504    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9d9247\""
Sep 15 21:27:53.914030 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.914007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:27:53.935110 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.935065    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:27:53.938982 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.938956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:53.941990 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.941968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9d9247-dockercfg-5rxkr\""
Sep 15 21:27:53.953875 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.953850    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:27:53.996551 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.996519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skbx4\" (UniqueName: \"kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:53.996727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.996670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:53.996727 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.996709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:53.996808 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:53.996737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75tnt\" (UniqueName: \"kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.097760 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.097728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbx4\" (UniqueName: \"kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:54.097955 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.097814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:54.097955 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.097847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.097955 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.097872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75tnt\" (UniqueName: \"kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.098674 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.098650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.098783 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.098648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:54.106687 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.106666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75tnt\" (UniqueName: \"kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt\") pod \"payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.106794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.106770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skbx4\" (UniqueName: \"kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4\") pod \"payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:54.212270 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.212180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:54.250723 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.250693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:54.488865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.488843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:27:54.491070 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:27:54.491042    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf4e6fd_38c1_4a9b_93f9_cf0375457199.slice/crio-2bc41c7ceb379982d1d6a3042737df83dd57800d71396641b14eaf1944c6265d WatchSource:0}: Error finding container 2bc41c7ceb379982d1d6a3042737df83dd57800d71396641b14eaf1944c6265d: Status 404 returned error can't find the container with id 2bc41c7ceb379982d1d6a3042737df83dd57800d71396641b14eaf1944c6265d
Sep 15 21:27:54.538769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:54.538743    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:27:54.541747 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:27:54.541711    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd32e80_10ef_4558_8edb_09b77bcf98e7.slice/crio-c0a4d4dcf812f617796ad8487bc15fae40bc7c0b3412098b2df599c61e4f6081 WatchSource:0}: Error finding container c0a4d4dcf812f617796ad8487bc15fae40bc7c0b3412098b2df599c61e4f6081: Status 404 returned error can't find the container with id c0a4d4dcf812f617796ad8487bc15fae40bc7c0b3412098b2df599c61e4f6081
Sep 15 21:27:55.058473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.058436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" event={"ID":"3cd32e80-10ef-4558-8edb-09b77bcf98e7","Type":"ContainerStarted","Data":"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"}
Sep 15 21:27:55.058992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.058503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" event={"ID":"3cd32e80-10ef-4558-8edb-09b77bcf98e7","Type":"ContainerStarted","Data":"c0a4d4dcf812f617796ad8487bc15fae40bc7c0b3412098b2df599c61e4f6081"}
Sep 15 21:27:55.058992 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.058689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:27:55.060194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.060169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" event={"ID":"9bf4e6fd-38c1-4a9b-93f9-cf0375457199","Type":"ContainerStarted","Data":"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"}
Sep 15 21:27:55.060331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.060219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:27:55.060331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.060243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" event={"ID":"9bf4e6fd-38c1-4a9b-93f9-cf0375457199","Type":"ContainerStarted","Data":"2bc41c7ceb379982d1d6a3042737df83dd57800d71396641b14eaf1944c6265d"}
Sep 15 21:27:55.077507 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.077460    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" podStartSLOduration=2.077440749 podStartE2EDuration="2.077440749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:27:55.076080337 +0000 UTC m=+1782.862448346" watchObservedRunningTime="2026-09-15 21:27:55.077440749 +0000 UTC m=+1782.863808758"
Sep 15 21:27:55.096176 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:27:55.096128    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" podStartSLOduration=2.096114742 podStartE2EDuration="2.096114742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:27:55.093955859 +0000 UTC m=+1782.880323869" watchObservedRunningTime="2026-09-15 21:27:55.096114742 +0000 UTC m=+1782.882482750"
Sep 15 21:28:06.069953 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:06.069875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:28:06.069953 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:06.069946    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:28:12.784394 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:12.784365    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:28:12.789630 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:12.789609    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:28:13.216776 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:13.216691    2577 scope.go:117] "RemoveContainer" containerID="de2949f1861578de69e64052a068535b7abf5e038f122e7b263e982af396f074"
Sep 15 21:28:20.469132 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.469099    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"]
Sep 15 21:28:20.472961 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.472936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:20.475878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.475853    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:28:20.476007 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.475853    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:28:20.482139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.482108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"]
Sep 15 21:28:20.572340 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.572304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck\" (UID: \"977e15ee-43b5-4e80-b3b8-1c0a23105696\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:20.673436 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.673396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck\" (UID: \"977e15ee-43b5-4e80-b3b8-1c0a23105696\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:20.674012 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.673993    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck\" (UID: \"977e15ee-43b5-4e80-b3b8-1c0a23105696\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:20.784899 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:20.784869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:21.059018 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:21.058989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"]
Sep 15 21:28:21.061004 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:28:21.060964    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977e15ee_43b5_4e80_b3b8_1c0a23105696.slice/crio-ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570 WatchSource:0}: Error finding container ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570: Status 404 returned error can't find the container with id ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570
Sep 15 21:28:21.222226 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:21.222187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck" event={"ID":"977e15ee-43b5-4e80-b3b8-1c0a23105696","Type":"ContainerStarted","Data":"ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570"}
Sep 15 21:28:22.230395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:22.230360    2577 generic.go:356] "Generic (PLEG): container finished" podID="977e15ee-43b5-4e80-b3b8-1c0a23105696" containerID="43f8b48eb7aa9486f44858d5a76ee391656da3bc0cb2de11775083c15417cea3" exitCode=0
Sep 15 21:28:22.230817 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:22.230450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck" event={"ID":"977e15ee-43b5-4e80-b3b8-1c0a23105696","Type":"ContainerDied","Data":"43f8b48eb7aa9486f44858d5a76ee391656da3bc0cb2de11775083c15417cea3"}
Sep 15 21:28:23.366250 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:23.366226    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:23.498777 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:23.498684    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca\") pod \"977e15ee-43b5-4e80-b3b8-1c0a23105696\" (UID: \"977e15ee-43b5-4e80-b3b8-1c0a23105696\") "
Sep 15 21:28:23.499023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:23.499001    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "977e15ee-43b5-4e80-b3b8-1c0a23105696" (UID: "977e15ee-43b5-4e80-b3b8-1c0a23105696"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:23.599972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:23.599942    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/977e15ee-43b5-4e80-b3b8-1c0a23105696-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:24.243433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:24.243407    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"
Sep 15 21:28:24.243634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:24.243451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck" event={"ID":"977e15ee-43b5-4e80-b3b8-1c0a23105696","Type":"ContainerDied","Data":"ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570"}
Sep 15 21:28:24.243634 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:24.243498    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed3a587676eb7cf7d43f7888810fa815bda7f33bed4ffc6ffd3ed17fb59fb570"
Sep 15 21:28:31.666082 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.666034    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:28:31.666580 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.666341    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" podUID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" containerName="payload-processing" containerID="cri-o://986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745" gracePeriod=30
Sep 15 21:28:31.674754 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.674722    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:28:31.676201 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.675172    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" podUID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" containerName="payload-pre-processing" containerID="cri-o://be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481" gracePeriod=30
Sep 15 21:28:31.844342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.844280    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"]
Sep 15 21:28:31.847844 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.847819    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9d9247-mgvck"]
Sep 15 21:28:31.972703 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.972675    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:28:31.993077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:31.993054    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:28:32.085579 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.085544    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skbx4\" (UniqueName: \"kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4\") pod \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") "
Sep 15 21:28:32.085776 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.085612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume\") pod \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") "
Sep 15 21:28:32.085776 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.085659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume\") pod \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\" (UID: \"9bf4e6fd-38c1-4a9b-93f9-cf0375457199\") "
Sep 15 21:28:32.085904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.085785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75tnt\" (UniqueName: \"kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt\") pod \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\" (UID: \"3cd32e80-10ef-4558-8edb-09b77bcf98e7\") "
Sep 15 21:28:32.086054 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.086029    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cd32e80-10ef-4558-8edb-09b77bcf98e7" (UID: "3cd32e80-10ef-4558-8edb-09b77bcf98e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:32.086131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.086034    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bf4e6fd-38c1-4a9b-93f9-cf0375457199" (UID: "9bf4e6fd-38c1-4a9b-93f9-cf0375457199"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:32.086233 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.086186    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd32e80-10ef-4558-8edb-09b77bcf98e7-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.087835 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.087813    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4" (OuterVolumeSpecName: "kube-api-access-skbx4") pod "9bf4e6fd-38c1-4a9b-93f9-cf0375457199" (UID: "9bf4e6fd-38c1-4a9b-93f9-cf0375457199"). InnerVolumeSpecName "kube-api-access-skbx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:32.087891 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.087814    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt" (OuterVolumeSpecName: "kube-api-access-75tnt") pod "3cd32e80-10ef-4558-8edb-09b77bcf98e7" (UID: "3cd32e80-10ef-4558-8edb-09b77bcf98e7"). InnerVolumeSpecName "kube-api-access-75tnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:32.187546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.187509    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.187546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.187540    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75tnt\" (UniqueName: \"kubernetes.io/projected/3cd32e80-10ef-4558-8edb-09b77bcf98e7-kube-api-access-75tnt\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.187546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.187552    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skbx4\" (UniqueName: \"kubernetes.io/projected/9bf4e6fd-38c1-4a9b-93f9-cf0375457199-kube-api-access-skbx4\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.289435 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.289404    2577 generic.go:356] "Generic (PLEG): container finished" podID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" containerID="986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745" exitCode=0
Sep 15 21:28:32.289655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.289450    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"
Sep 15 21:28:32.289655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.289489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" event={"ID":"3cd32e80-10ef-4558-8edb-09b77bcf98e7","Type":"ContainerDied","Data":"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"}
Sep 15 21:28:32.289655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.289539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k" event={"ID":"3cd32e80-10ef-4558-8edb-09b77bcf98e7","Type":"ContainerDied","Data":"c0a4d4dcf812f617796ad8487bc15fae40bc7c0b3412098b2df599c61e4f6081"}
Sep 15 21:28:32.289655 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.289598    2577 scope.go:117] "RemoveContainer" containerID="986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"
Sep 15 21:28:32.291186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.291162    2577 generic.go:356] "Generic (PLEG): container finished" podID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" containerID="be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481" exitCode=0
Sep 15 21:28:32.291307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.291209    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"
Sep 15 21:28:32.291307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.291229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" event={"ID":"9bf4e6fd-38c1-4a9b-93f9-cf0375457199","Type":"ContainerDied","Data":"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"}
Sep 15 21:28:32.291307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.291288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r" event={"ID":"9bf4e6fd-38c1-4a9b-93f9-cf0375457199","Type":"ContainerDied","Data":"2bc41c7ceb379982d1d6a3042737df83dd57800d71396641b14eaf1944c6265d"}
Sep 15 21:28:32.299111 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.299082    2577 scope.go:117] "RemoveContainer" containerID="986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"
Sep 15 21:28:32.299467 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:28:32.299442    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745\": container with ID starting with 986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745 not found: ID does not exist" containerID="986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"
Sep 15 21:28:32.299663 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.299472    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745"} err="failed to get container status \"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745\": rpc error: code = NotFound desc = could not find container \"986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745\": container with ID starting with 986a43292a1d24f1a35f3b6f373388fe4ca875eb71926e2bb0756f4354cd9745 not found: ID does not exist"
Sep 15 21:28:32.299663 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.299488    2577 scope.go:117] "RemoveContainer" containerID="be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"
Sep 15 21:28:32.308373 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.308356    2577 scope.go:117] "RemoveContainer" containerID="be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"
Sep 15 21:28:32.308681 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:28:32.308662    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481\": container with ID starting with be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481 not found: ID does not exist" containerID="be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"
Sep 15 21:28:32.308745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.308690    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481"} err="failed to get container status \"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481\": rpc error: code = NotFound desc = could not find container \"be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481\": container with ID starting with be13005f72299d72ffca65962e1e764d639f10449c05af7ec0e717032f8de481 not found: ID does not exist"
Sep 15 21:28:32.316938 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.316915    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:28:32.323042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.323014    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9d9247-6544f8fdd6-zfg9r"]
Sep 15 21:28:32.333783 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.333761    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:28:32.337775 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.337756    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9d9247-6ccd8bdcfc-pnc4k"]
Sep 15 21:28:32.692279 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.692251    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" path="/var/lib/kubelet/pods/3cd32e80-10ef-4558-8edb-09b77bcf98e7/volumes"
Sep 15 21:28:32.692619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.692554    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="977e15ee-43b5-4e80-b3b8-1c0a23105696" path="/var/lib/kubelet/pods/977e15ee-43b5-4e80-b3b8-1c0a23105696/volumes"
Sep 15 21:28:32.692936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:32.692921    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" path="/var/lib/kubelet/pods/9bf4e6fd-38c1-4a9b-93f9-cf0375457199/volumes"
Sep 15 21:28:41.724531 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.724495    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"]
Sep 15 21:28:41.724999 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.724978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" containerName="payload-processing"
Sep 15 21:28:41.724999 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.724992    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" containerName="payload-processing"
Sep 15 21:28:41.725077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725003    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" containerName="payload-pre-processing"
Sep 15 21:28:41.725077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725008    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" containerName="payload-pre-processing"
Sep 15 21:28:41.725077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725024    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="977e15ee-43b5-4e80-b3b8-1c0a23105696" containerName="revoke-keys"
Sep 15 21:28:41.725077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725032    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="977e15ee-43b5-4e80-b3b8-1c0a23105696" containerName="revoke-keys"
Sep 15 21:28:41.725201 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725118    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cd32e80-10ef-4558-8edb-09b77bcf98e7" containerName="payload-processing"
Sep 15 21:28:41.725201 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725125    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="977e15ee-43b5-4e80-b3b8-1c0a23105696" containerName="revoke-keys"
Sep 15 21:28:41.725201 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.725132    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf4e6fd-38c1-4a9b-93f9-cf0375457199" containerName="payload-pre-processing"
Sep 15 21:28:41.729833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.729817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:41.733128 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.732818    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:28:41.733128 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.732959    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:28:41.734935 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.734909    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"]
Sep 15 21:28:41.887139 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.887105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh\" (UID: \"ab24d7c1-a373-40aa-b056-153941df8fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:41.988243 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.988217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh\" (UID: \"ab24d7c1-a373-40aa-b056-153941df8fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:41.988941 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:41.988917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh\" (UID: \"ab24d7c1-a373-40aa-b056-153941df8fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:42.043710 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:42.043677    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:42.323596 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:42.323552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"]
Sep 15 21:28:42.325636 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:28:42.325606    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab24d7c1_a373_40aa_b056_153941df8fb6.slice/crio-c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e WatchSource:0}: Error finding container c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e: Status 404 returned error can't find the container with id c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e
Sep 15 21:28:42.352495 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:42.352459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh" event={"ID":"ab24d7c1-a373-40aa-b056-153941df8fb6","Type":"ContainerStarted","Data":"c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e"}
Sep 15 21:28:43.359342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:43.359306    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab24d7c1-a373-40aa-b056-153941df8fb6" containerID="10eeb99bd5cc47e0cfd1174d706bec11e0456d67833ec9afab69eff7a486adac" exitCode=0
Sep 15 21:28:43.359342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:43.359346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh" event={"ID":"ab24d7c1-a373-40aa-b056-153941df8fb6","Type":"ContainerDied","Data":"10eeb99bd5cc47e0cfd1174d706bec11e0456d67833ec9afab69eff7a486adac"}
Sep 15 21:28:44.511389 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:44.511365    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:44.710869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:44.710787    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca\") pod \"ab24d7c1-a373-40aa-b056-153941df8fb6\" (UID: \"ab24d7c1-a373-40aa-b056-153941df8fb6\") "
Sep 15 21:28:44.711155 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:44.711132    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab24d7c1-a373-40aa-b056-153941df8fb6" (UID: "ab24d7c1-a373-40aa-b056-153941df8fb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:44.812146 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:44.812118    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab24d7c1-a373-40aa-b056-153941df8fb6-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:45.373931 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:45.373902    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"
Sep 15 21:28:45.374107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:45.373940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh" event={"ID":"ab24d7c1-a373-40aa-b056-153941df8fb6","Type":"ContainerDied","Data":"c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e"}
Sep 15 21:28:45.374107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:45.373989    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6bf48d10f57da672be1d0085fb3fd30c9b061a76dbb687d8c5341aff3a7993e"
Sep 15 21:28:57.441011 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.440977    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:28:57.441513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.441229    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" podUID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" containerName="payload-pre-processing" containerID="cri-o://2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35" gracePeriod=30
Sep 15 21:28:57.620153 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.620124    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"]
Sep 15 21:28:57.622318 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.622289    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b8f396-2xzdh"]
Sep 15 21:28:57.720858 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.720819    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:28:57.842659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.842629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8prcf\" (UniqueName: \"kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf\") pod \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") "
Sep 15 21:28:57.842848 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.842684    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume\") pod \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\" (UID: \"fd9204c6-087d-4eb1-ad2b-abce7cdea605\") "
Sep 15 21:28:57.843119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.843093    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd9204c6-087d-4eb1-ad2b-abce7cdea605" (UID: "fd9204c6-087d-4eb1-ad2b-abce7cdea605"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:57.845034 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.845011    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf" (OuterVolumeSpecName: "kube-api-access-8prcf") pod "fd9204c6-087d-4eb1-ad2b-abce7cdea605" (UID: "fd9204c6-087d-4eb1-ad2b-abce7cdea605"). InnerVolumeSpecName "kube-api-access-8prcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:57.944581 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.944535    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8prcf\" (UniqueName: \"kubernetes.io/projected/fd9204c6-087d-4eb1-ad2b-abce7cdea605-kube-api-access-8prcf\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.944748 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:57.944611    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9204c6-087d-4eb1-ad2b-abce7cdea605-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:28:58.455529 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.455497    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" containerID="2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35" exitCode=0
Sep 15 21:28:58.455995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.455548    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"
Sep 15 21:28:58.455995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.455598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" event={"ID":"fd9204c6-087d-4eb1-ad2b-abce7cdea605","Type":"ContainerDied","Data":"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"}
Sep 15 21:28:58.455995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.455655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7" event={"ID":"fd9204c6-087d-4eb1-ad2b-abce7cdea605","Type":"ContainerDied","Data":"db7a447d9e2eaca1096b9f26d8aca3b2ae5c464ccf5bf71b4b795dc6a678ad71"}
Sep 15 21:28:58.455995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.455679    2577 scope.go:117] "RemoveContainer" containerID="2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"
Sep 15 21:28:58.465045 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.465029    2577 scope.go:117] "RemoveContainer" containerID="2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"
Sep 15 21:28:58.465312 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:28:58.465294    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35\": container with ID starting with 2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35 not found: ID does not exist" containerID="2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"
Sep 15 21:28:58.465365 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.465320    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35"} err="failed to get container status \"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35\": rpc error: code = NotFound desc = could not find container \"2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35\": container with ID starting with 2e50f6d06503ae4faffa55d0a710a5fae75a53f17bb1863ec733aaffd90a2a35 not found: ID does not exist"
Sep 15 21:28:58.482662 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.482637    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:28:58.489470 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.489449    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f396-5d6f647fb6-xc6s7"]
Sep 15 21:28:58.691747 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.691705    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab24d7c1-a373-40aa-b056-153941df8fb6" path="/var/lib/kubelet/pods/ab24d7c1-a373-40aa-b056-153941df8fb6/volumes"
Sep 15 21:28:58.692342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:28:58.692277    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" path="/var/lib/kubelet/pods/fd9204c6-087d-4eb1-ad2b-abce7cdea605/volumes"
Sep 15 21:29:13.238158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:13.238122    2577 scope.go:117] "RemoveContainer" containerID="7d75deb934c21530095c754ce970500ef9a72f1c1cd4573f9dbfe1bf533c84f3"
Sep 15 21:29:13.247108 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:13.247086    2577 scope.go:117] "RemoveContainer" containerID="1a2c08117bd553e6fc0f356d098cab8fd5fa262c9ee5280c0a408e2ba4917a8e"
Sep 15 21:29:13.254854 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:13.254835    2577 scope.go:117] "RemoveContainer" containerID="dc04c135fff84cea03cc4f7ff96801eb8df99ca618f1a8fa03223662db0a03f8"
Sep 15 21:29:53.465307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.465223    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:29:53.465911 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.465489    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy" containerID="cri-o://cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16" gracePeriod=30
Sep 15 21:29:53.819633 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.819599    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"]
Sep 15 21:29:53.820066 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820051    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" containerName="payload-pre-processing"
Sep 15 21:29:53.820126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820069    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" containerName="payload-pre-processing"
Sep 15 21:29:53.820126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820090    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab24d7c1-a373-40aa-b056-153941df8fb6" containerName="revoke-keys"
Sep 15 21:29:53.820126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820100    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab24d7c1-a373-40aa-b056-153941df8fb6" containerName="revoke-keys"
Sep 15 21:29:53.820226 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820174    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab24d7c1-a373-40aa-b056-153941df8fb6" containerName="revoke-keys"
Sep 15 21:29:53.820226 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.820188    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd9204c6-087d-4eb1-ad2b-abce7cdea605" containerName="payload-pre-processing"
Sep 15 21:29:53.823475 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.823457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:53.826119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.826100    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:29:53.826119 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.826110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:29:53.831695 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:53.831651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"]
Sep 15 21:29:54.002169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.002138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg\" (UID: \"c4cc457f-a7c4-4a5e-ae2f-b8c274652319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:54.103225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.103133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg\" (UID: \"c4cc457f-a7c4-4a5e-ae2f-b8c274652319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:54.103806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.103787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg\" (UID: \"c4cc457f-a7c4-4a5e-ae2f-b8c274652319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:54.135889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.135856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:54.408117 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.408091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"]
Sep 15 21:29:54.410694 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:29:54.410668    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cc457f_a7c4_4a5e_ae2f_b8c274652319.slice/crio-eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9 WatchSource:0}: Error finding container eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9: Status 404 returned error can't find the container with id eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9
Sep 15 21:29:54.769415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:54.769382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg" event={"ID":"c4cc457f-a7c4-4a5e-ae2f-b8c274652319","Type":"ContainerStarted","Data":"eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9"}
Sep 15 21:29:55.776214 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:55.776176    2577 generic.go:356] "Generic (PLEG): container finished" podID="c4cc457f-a7c4-4a5e-ae2f-b8c274652319" containerID="14dd36585e72d11be2d9051810601ba99be01cfefd052cb9edadd4a54520aa8e" exitCode=0
Sep 15 21:29:55.776605 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:55.776265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg" event={"ID":"c4cc457f-a7c4-4a5e-ae2f-b8c274652319","Type":"ContainerDied","Data":"14dd36585e72d11be2d9051810601ba99be01cfefd052cb9edadd4a54520aa8e"}
Sep 15 21:29:56.930904 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:56.930880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:56.931952 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:56.931930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca\") pod \"c4cc457f-a7c4-4a5e-ae2f-b8c274652319\" (UID: \"c4cc457f-a7c4-4a5e-ae2f-b8c274652319\") "
Sep 15 21:29:56.932257 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:56.932238    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c4cc457f-a7c4-4a5e-ae2f-b8c274652319" (UID: "c4cc457f-a7c4-4a5e-ae2f-b8c274652319"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:29:57.033400 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:57.033310    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4cc457f-a7c4-4a5e-ae2f-b8c274652319-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:57.788865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:57.788832    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"
Sep 15 21:29:57.788865 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:57.788850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg" event={"ID":"c4cc457f-a7c4-4a5e-ae2f-b8c274652319","Type":"ContainerDied","Data":"eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9"}
Sep 15 21:29:57.789086 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:57.788899    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaf05450b317b9abae9818cc3007978651cccae919f0fd4a73bce12235a221a9"
Sep 15 21:29:58.721415 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.721386    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:29:58.796149 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.796118    2577 generic.go:356] "Generic (PLEG): container finished" podID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerID="cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16" exitCode=0
Sep 15 21:29:58.796294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.796165    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"
Sep 15 21:29:58.796294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.796207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerDied","Data":"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"}
Sep 15 21:29:58.796294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.796263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw" event={"ID":"08dde8f1-ae31-4a24-b219-21efcd5ed514","Type":"ContainerDied","Data":"64ec92581ffcb6ce16f35372170393bf17206906adbe7e21cd442ea488c5db96"}
Sep 15 21:29:58.796294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.796290    2577 scope.go:117] "RemoveContainer" containerID="cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"
Sep 15 21:29:58.805489 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.805471    2577 scope.go:117] "RemoveContainer" containerID="eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"
Sep 15 21:29:58.813840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.813822    2577 scope.go:117] "RemoveContainer" containerID="cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"
Sep 15 21:29:58.814116 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:29:58.814102    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16\": container with ID starting with cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16 not found: ID does not exist" containerID="cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"
Sep 15 21:29:58.814169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.814125    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16"} err="failed to get container status \"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16\": rpc error: code = NotFound desc = could not find container \"cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16\": container with ID starting with cb5981ea1864ae7ef07612209682b1f859014adcd29522113cb1ff2db0869f16 not found: ID does not exist"
Sep 15 21:29:58.814169 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.814141    2577 scope.go:117] "RemoveContainer" containerID="eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"
Sep 15 21:29:58.814394 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:29:58.814378    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63\": container with ID starting with eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63 not found: ID does not exist" containerID="eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"
Sep 15 21:29:58.814438 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.814400    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63"} err="failed to get container status \"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63\": rpc error: code = NotFound desc = could not find container \"eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63\": container with ID starting with eefc32068abc696b008892465d7402508b30f6521db321199323ac63e9f46b63 not found: ID does not exist"
Sep 15 21:29:58.847546 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847599    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847720    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25kq9\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.847821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847807    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data" (OuterVolumeSpecName: "istio-data") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:58.848026 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.848026 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.847918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs\") pod \"08dde8f1-ae31-4a24-b219-21efcd5ed514\" (UID: \"08dde8f1-ae31-4a24-b219-21efcd5ed514\") "
Sep 15 21:29:58.848133 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:58.848202 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:58.848427 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848287    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:58.848725 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:29:58.848806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848757    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.848806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848777    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.848806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848789    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.848806 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848799    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.848949 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.848811    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08dde8f1-ae31-4a24-b219-21efcd5ed514-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.850095 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.850070    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:58.850182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.850119    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9" (OuterVolumeSpecName: "kube-api-access-25kq9") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "kube-api-access-25kq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:29:58.850353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.850334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:29:58.850353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.850346    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token" (OuterVolumeSpecName: "istio-token") pod "08dde8f1-ae31-4a24-b219-21efcd5ed514" (UID: "08dde8f1-ae31-4a24-b219-21efcd5ed514"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:29:58.949903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.949873    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25kq9\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-kube-api-access-25kq9\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.949903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.949900    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.949903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.949909    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:58.950133 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:58.949917    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08dde8f1-ae31-4a24-b219-21efcd5ed514-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:29:59.121364 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:59.121331    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:29:59.126525 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:29:59.126501    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-b8f396-openshift-default-76c59d7446-kvbvw"]
Sep 15 21:30:00.178987 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.178953    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"]
Sep 15 21:30:00.179423 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179410    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.179423 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179424    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.179506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179431    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.179506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.179506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179466    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4cc457f-a7c4-4a5e-ae2f-b8c274652319" containerName="revoke-keys"
Sep 15 21:30:00.179506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179472    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4cc457f-a7c4-4a5e-ae2f-b8c274652319" containerName="revoke-keys"
Sep 15 21:30:00.179682 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179529    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.179682 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179536    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4cc457f-a7c4-4a5e-ae2f-b8c274652319" containerName="revoke-keys"
Sep 15 21:30:00.179682 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.179546    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" containerName="istio-proxy"
Sep 15 21:30:00.184131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.184116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"
Sep 15 21:30:00.187437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.187414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:30:00.193970 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.193953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"
Sep 15 21:30:00.208194 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.208144    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"]
Sep 15 21:30:00.212541 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.212517    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"]
Sep 15 21:30:00.212697 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.212662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"
Sep 15 21:30:00.214079 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.214061    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"]
Sep 15 21:30:00.217821 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.217804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"
Sep 15 21:30:00.233756 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.233616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"]
Sep 15 21:30:00.236248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.236222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"
Sep 15 21:30:00.244915 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.244898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"
Sep 15 21:30:00.269097 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.267794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"]
Sep 15 21:30:00.486795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.486768    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"]
Sep 15 21:30:00.488119 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:30:00.488090    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93dabb3b_0a0e_4431_95c1_a55271d389e6.slice/crio-3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2 WatchSource:0}: Error finding container 3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2: Status 404 returned error can't find the container with id 3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2
Sep 15 21:30:00.533116 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.533086    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"]
Sep 15 21:30:00.534310 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:30:00.534289    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bed977c_ff98_4f7f_9101_32b9d73f9e81.slice/crio-52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f WatchSource:0}: Error finding container 52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f: Status 404 returned error can't find the container with id 52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f
Sep 15 21:30:00.550041 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.550020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"]
Sep 15 21:30:00.551721 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:30:00.551687    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca474096_0847_4f4b_ae20_d08130f8b073.slice/crio-71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2 WatchSource:0}: Error finding container 71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2: Status 404 returned error can't find the container with id 71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2
Sep 15 21:30:00.692311 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.692281    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08dde8f1-ae31-4a24-b219-21efcd5ed514" path="/var/lib/kubelet/pods/08dde8f1-ae31-4a24-b219-21efcd5ed514/volumes"
Sep 15 21:30:00.809506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.809468    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn" event={"ID":"2bed977c-ff98-4f7f-9101-32b9d73f9e81","Type":"ContainerStarted","Data":"52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f"}
Sep 15 21:30:00.810720 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.810698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh" event={"ID":"93dabb3b-0a0e-4431-95c1-a55271d389e6","Type":"ContainerStarted","Data":"3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2"}
Sep 15 21:30:00.811903 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:00.811876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9" event={"ID":"ca474096-0847-4f4b-ae20-d08130f8b073","Type":"ContainerStarted","Data":"71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2"}
Sep 15 21:30:01.818789 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.818756    2577 generic.go:356] "Generic (PLEG): container finished" podID="2bed977c-ff98-4f7f-9101-32b9d73f9e81" containerID="2e424f98cefff14b1e9c556e8d079b44f55c593f229ae7881c8d4f6203d69422" exitCode=0
Sep 15 21:30:01.819217 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.818841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn" event={"ID":"2bed977c-ff98-4f7f-9101-32b9d73f9e81","Type":"ContainerDied","Data":"2e424f98cefff14b1e9c556e8d079b44f55c593f229ae7881c8d4f6203d69422"}
Sep 15 21:30:01.820403 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.820384    2577 generic.go:356] "Generic (PLEG): container finished" podID="93dabb3b-0a0e-4431-95c1-a55271d389e6" containerID="03f0f44bd35bb3a893aa24ec2fc279cdc9f108350ee80b36ab54ab742ef732ae" exitCode=0
Sep 15 21:30:01.820506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.820455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh" event={"ID":"93dabb3b-0a0e-4431-95c1-a55271d389e6","Type":"ContainerDied","Data":"03f0f44bd35bb3a893aa24ec2fc279cdc9f108350ee80b36ab54ab742ef732ae"}
Sep 15 21:30:01.821974 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.821953    2577 generic.go:356] "Generic (PLEG): container finished" podID="ca474096-0847-4f4b-ae20-d08130f8b073" containerID="bc440888b9d61bee480c6dc786861efc4a14465128b2a779197c4fd0257ca1d1" exitCode=0
Sep 15 21:30:01.822087 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:01.821976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9" event={"ID":"ca474096-0847-4f4b-ae20-d08130f8b073","Type":"ContainerDied","Data":"bc440888b9d61bee480c6dc786861efc4a14465128b2a779197c4fd0257ca1d1"}
Sep 15 21:30:02.974619 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:02.974584    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"
Sep 15 21:30:02.999495 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:02.999470    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"
Sep 15 21:30:03.035689 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.035663    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"
Sep 15 21:30:03.834960 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.834934    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9"
Sep 15 21:30:03.835208 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.834986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825130-scbz9" event={"ID":"ca474096-0847-4f4b-ae20-d08130f8b073","Type":"ContainerDied","Data":"71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2"}
Sep 15 21:30:03.835208 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.835048    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71c0908bb66408274ebd6977f2c667c871566c644facbb7a167c45b04e29dab2"
Sep 15 21:30:03.836609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.836558    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"
Sep 15 21:30:03.836609 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.836602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn" event={"ID":"2bed977c-ff98-4f7f-9101-32b9d73f9e81","Type":"ContainerDied","Data":"52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f"}
Sep 15 21:30:03.836797 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.836644    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52c69fa02b819f7810e751729feef93c1b20518d28c237f290aa9a02af26d78f"
Sep 15 21:30:03.838093 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.838067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh" event={"ID":"93dabb3b-0a0e-4431-95c1-a55271d389e6","Type":"ContainerDied","Data":"3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2"}
Sep 15 21:30:03.838190 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.838118    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3511bfadc7ce1466c103f37ae906fa0871e06600d74abf6dec7270f4913f8de2"
Sep 15 21:30:03.838190 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:03.838079    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"
Sep 15 21:30:04.576582 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.576538    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"]
Sep 15 21:30:04.591338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.591302    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ffb334-29825130-l4lmn"]
Sep 15 21:30:04.617468 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.617433    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:30:04.617794 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.617765    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" podUID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" containerName="payload-processing" containerID="cri-o://b3984f13e2b711a78e9dde9ead47ee2b01afdee8d9d2a8dc5301e8c1f7b392ad" gracePeriod=30
Sep 15 21:30:04.623457 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.623427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:30:04.623815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.623688    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" podUID="15c6ca6a-08cb-4823-abce-c570f46c5886" containerName="payload-pre-processing" containerID="cri-o://de827feb10f4df0fca0a5ea9371f9cab802935f0b2bf8eed83cd85e767790b01" gracePeriod=30
Sep 15 21:30:04.692059 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.692029    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bed977c-ff98-4f7f-9101-32b9d73f9e81" path="/var/lib/kubelet/pods/2bed977c-ff98-4f7f-9101-32b9d73f9e81/volumes"
Sep 15 21:30:04.837313 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.837208    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"]
Sep 15 21:30:04.843736 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.843694    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ffb334-v65dg"]
Sep 15 21:30:04.849077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.849041    2577 generic.go:356] "Generic (PLEG): container finished" podID="15c6ca6a-08cb-4823-abce-c570f46c5886" containerID="de827feb10f4df0fca0a5ea9371f9cab802935f0b2bf8eed83cd85e767790b01" exitCode=0
Sep 15 21:30:04.849227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.849188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" event={"ID":"15c6ca6a-08cb-4823-abce-c570f46c5886","Type":"ContainerDied","Data":"de827feb10f4df0fca0a5ea9371f9cab802935f0b2bf8eed83cd85e767790b01"}
Sep 15 21:30:04.851495 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.851451    2577 generic.go:356] "Generic (PLEG): container finished" podID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" containerID="b3984f13e2b711a78e9dde9ead47ee2b01afdee8d9d2a8dc5301e8c1f7b392ad" exitCode=0
Sep 15 21:30:04.851728 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.851635    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" event={"ID":"20f5b470-594e-4c1e-98c6-ff0daee88ff4","Type":"ContainerDied","Data":"b3984f13e2b711a78e9dde9ead47ee2b01afdee8d9d2a8dc5301e8c1f7b392ad"}
Sep 15 21:30:04.916849 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.916827    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:30:04.919797 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:04.919778    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:30:05.013484 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.013449    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn2d4\" (UniqueName: \"kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4\") pod \"15c6ca6a-08cb-4823-abce-c570f46c5886\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") "
Sep 15 21:30:05.013693 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.013543    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume\") pod \"15c6ca6a-08cb-4823-abce-c570f46c5886\" (UID: \"15c6ca6a-08cb-4823-abce-c570f46c5886\") "
Sep 15 21:30:05.013693 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.013608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svzgz\" (UniqueName: \"kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz\") pod \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") "
Sep 15 21:30:05.013693 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.013646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume\") pod \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\" (UID: \"20f5b470-594e-4c1e-98c6-ff0daee88ff4\") "
Sep 15 21:30:05.014036 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.013932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15c6ca6a-08cb-4823-abce-c570f46c5886" (UID: "15c6ca6a-08cb-4823-abce-c570f46c5886"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:05.014156 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.014058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20f5b470-594e-4c1e-98c6-ff0daee88ff4" (UID: "20f5b470-594e-4c1e-98c6-ff0daee88ff4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:05.015683 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.015659    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4" (OuterVolumeSpecName: "kube-api-access-dn2d4") pod "15c6ca6a-08cb-4823-abce-c570f46c5886" (UID: "15c6ca6a-08cb-4823-abce-c570f46c5886"). InnerVolumeSpecName "kube-api-access-dn2d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:05.015775 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.015739    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz" (OuterVolumeSpecName: "kube-api-access-svzgz") pod "20f5b470-594e-4c1e-98c6-ff0daee88ff4" (UID: "20f5b470-594e-4c1e-98c6-ff0daee88ff4"). InnerVolumeSpecName "kube-api-access-svzgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:05.114545 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.114468    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c6ca6a-08cb-4823-abce-c570f46c5886-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:05.114545 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.114492    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svzgz\" (UniqueName: \"kubernetes.io/projected/20f5b470-594e-4c1e-98c6-ff0daee88ff4-kube-api-access-svzgz\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:05.114545 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.114502    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f5b470-594e-4c1e-98c6-ff0daee88ff4-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:05.114545 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.114510    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn2d4\" (UniqueName: \"kubernetes.io/projected/15c6ca6a-08cb-4823-abce-c570f46c5886-kube-api-access-dn2d4\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:05.859617 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.859586    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"
Sep 15 21:30:05.859617 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.859606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w" event={"ID":"20f5b470-594e-4c1e-98c6-ff0daee88ff4","Type":"ContainerDied","Data":"144d99bc779cdfad3c7a81750dfc1617bb89790ef9b30a4484b58864b1623505"}
Sep 15 21:30:05.860136 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.859679    2577 scope.go:117] "RemoveContainer" containerID="b3984f13e2b711a78e9dde9ead47ee2b01afdee8d9d2a8dc5301e8c1f7b392ad"
Sep 15 21:30:05.861370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.861341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps" event={"ID":"15c6ca6a-08cb-4823-abce-c570f46c5886","Type":"ContainerDied","Data":"98e59dc14372e8a6ad3f7b087a126ecd0dff34ed36ee6ceccaac380928c44d7d"}
Sep 15 21:30:05.861462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.861351    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"
Sep 15 21:30:05.870191 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.870170    2577 scope.go:117] "RemoveContainer" containerID="de827feb10f4df0fca0a5ea9371f9cab802935f0b2bf8eed83cd85e767790b01"
Sep 15 21:30:05.883459 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.883434    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:30:05.889405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.889384    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ffb334-fff6bbfd7-phv8w"]
Sep 15 21:30:05.901359 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.901335    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:30:05.907779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:05.907755    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ffb334-59575748cd-lgnps"]
Sep 15 21:30:06.690985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:06.690956    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c6ca6a-08cb-4823-abce-c570f46c5886" path="/var/lib/kubelet/pods/15c6ca6a-08cb-4823-abce-c570f46c5886/volumes"
Sep 15 21:30:06.691320 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:06.691308    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" path="/var/lib/kubelet/pods/20f5b470-594e-4c1e-98c6-ff0daee88ff4/volumes"
Sep 15 21:30:06.691623 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:06.691611    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4cc457f-a7c4-4a5e-ae2f-b8c274652319" path="/var/lib/kubelet/pods/c4cc457f-a7c4-4a5e-ae2f-b8c274652319/volumes"
Sep 15 21:30:11.880815 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.880783    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"]
Sep 15 21:30:11.881310 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881292    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93dabb3b-0a0e-4431-95c1-a55271d389e6" containerName="cleanup"
Sep 15 21:30:11.881310 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881311    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="93dabb3b-0a0e-4431-95c1-a55271d389e6" containerName="cleanup"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881329    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" containerName="payload-processing"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881335    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" containerName="payload-processing"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881347    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bed977c-ff98-4f7f-9101-32b9d73f9e81" containerName="cleanup"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881356    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bed977c-ff98-4f7f-9101-32b9d73f9e81" containerName="cleanup"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881366    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c6ca6a-08cb-4823-abce-c570f46c5886" containerName="payload-pre-processing"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881371    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c6ca6a-08cb-4823-abce-c570f46c5886" containerName="payload-pre-processing"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881380    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca474096-0847-4f4b-ae20-d08130f8b073" containerName="cleanup"
Sep 15 21:30:11.881432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881385    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca474096-0847-4f4b-ae20-d08130f8b073" containerName="cleanup"
Sep 15 21:30:11.881881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881462    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="93dabb3b-0a0e-4431-95c1-a55271d389e6" containerName="cleanup"
Sep 15 21:30:11.881881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881471    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c6ca6a-08cb-4823-abce-c570f46c5886" containerName="payload-pre-processing"
Sep 15 21:30:11.881881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881477    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca474096-0847-4f4b-ae20-d08130f8b073" containerName="cleanup"
Sep 15 21:30:11.881881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881485    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bed977c-ff98-4f7f-9101-32b9d73f9e81" containerName="cleanup"
Sep 15 21:30:11.881881 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.881493    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f5b470-594e-4c1e-98c6-ff0daee88ff4" containerName="payload-processing"
Sep 15 21:30:11.886112 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.886095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:11.888880 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.888859    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:30:11.888995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.888866    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:30:11.893523 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.893487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"]
Sep 15 21:30:11.985240 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:11.985210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9\" (UID: \"3583087f-08d1-4d7b-9aa2-a239b5a76d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:12.086387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.086355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9\" (UID: \"3583087f-08d1-4d7b-9aa2-a239b5a76d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:12.086988 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.086968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9\" (UID: \"3583087f-08d1-4d7b-9aa2-a239b5a76d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:12.198178 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.198095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:12.477520 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.477497    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"]
Sep 15 21:30:12.479583 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:30:12.479538    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3583087f_08d1_4d7b_9aa2_a239b5a76d30.slice/crio-6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1 WatchSource:0}: Error finding container 6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1: Status 404 returned error can't find the container with id 6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1
Sep 15 21:30:12.907310 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.907274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9" event={"ID":"3583087f-08d1-4d7b-9aa2-a239b5a76d30","Type":"ContainerStarted","Data":"727e317ed5830a96034c3ab9ff2c76e4105ccd3f192a9433e22aa7cc033156ff"}
Sep 15 21:30:12.907745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.907333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9" event={"ID":"3583087f-08d1-4d7b-9aa2-a239b5a76d30","Type":"ContainerStarted","Data":"6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1"}
Sep 15 21:30:12.947628 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:12.947578    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9" podStartSLOduration=1.947544693 podStartE2EDuration="1.947544693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:30:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:30:12.945409689 +0000 UTC m=+1920.731777699" watchObservedRunningTime="2026-09-15 21:30:12.947544693 +0000 UTC m=+1920.733912700"
Sep 15 21:30:13.293405 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:13.293375    2577 scope.go:117] "RemoveContainer" containerID="6c67c01962220d00f9c0ed1cc0cd1e0c98814fa2d089630c5d8fd3c6d3390f11"
Sep 15 21:30:13.913986 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:13.913951    2577 generic.go:356] "Generic (PLEG): container finished" podID="3583087f-08d1-4d7b-9aa2-a239b5a76d30" containerID="727e317ed5830a96034c3ab9ff2c76e4105ccd3f192a9433e22aa7cc033156ff" exitCode=0
Sep 15 21:30:13.914366 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:13.914039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9" event={"ID":"3583087f-08d1-4d7b-9aa2-a239b5a76d30","Type":"ContainerDied","Data":"727e317ed5830a96034c3ab9ff2c76e4105ccd3f192a9433e22aa7cc033156ff"}
Sep 15 21:30:15.062892 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.062866    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:15.115338 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.115298    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca\") pod \"3583087f-08d1-4d7b-9aa2-a239b5a76d30\" (UID: \"3583087f-08d1-4d7b-9aa2-a239b5a76d30\") "
Sep 15 21:30:15.115698 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.115670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3583087f-08d1-4d7b-9aa2-a239b5a76d30" (UID: "3583087f-08d1-4d7b-9aa2-a239b5a76d30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:15.217052 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.216973    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3583087f-08d1-4d7b-9aa2-a239b5a76d30-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:15.925990 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.925963    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"
Sep 15 21:30:15.925990 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.925980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9" event={"ID":"3583087f-08d1-4d7b-9aa2-a239b5a76d30","Type":"ContainerDied","Data":"6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1"}
Sep 15 21:30:15.926189 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:15.926024    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b35073f13c96ad8102b3aa8506e9069a7e705bc6880621c57363e10383c44b1"
Sep 15 21:30:22.573769 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.573724    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:30:22.574324 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.573937    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" podUID="3366106e-cdc7-4b50-b8f3-7e144439fc75" containerName="payload-pre-processing" containerID="cri-o://2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c" gracePeriod=30
Sep 15 21:30:22.738133 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.738098    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"]
Sep 15 21:30:22.739959 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.739930    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2f34ef-qrbg9"]
Sep 15 21:30:22.833301 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.833241    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:30:22.971362 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.971325    2577 generic.go:356] "Generic (PLEG): container finished" podID="3366106e-cdc7-4b50-b8f3-7e144439fc75" containerID="2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c" exitCode=0
Sep 15 21:30:22.971521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.971373    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"
Sep 15 21:30:22.971521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.971375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" event={"ID":"3366106e-cdc7-4b50-b8f3-7e144439fc75","Type":"ContainerDied","Data":"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"}
Sep 15 21:30:22.971521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.971482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5" event={"ID":"3366106e-cdc7-4b50-b8f3-7e144439fc75","Type":"ContainerDied","Data":"62a9fa4f41ef0f235608deaca134d559360de27e4fcbce8fc16e93ee4f1d7f40"}
Sep 15 21:30:22.971521 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.971510    2577 scope.go:117] "RemoveContainer" containerID="2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"
Sep 15 21:30:22.980084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.980050    2577 scope.go:117] "RemoveContainer" containerID="2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"
Sep 15 21:30:22.980377 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:30:22.980357    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c\": container with ID starting with 2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c not found: ID does not exist" containerID="2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"
Sep 15 21:30:22.980447 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.980387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c"} err="failed to get container status \"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c\": rpc error: code = NotFound desc = could not find container \"2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c\": container with ID starting with 2c9829a2530a76385b1540ae34df17fbde623d5e3f10bc4a5a8d2db0c645948c not found: ID does not exist"
Sep 15 21:30:22.989759 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.989741    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume\") pod \"3366106e-cdc7-4b50-b8f3-7e144439fc75\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") "
Sep 15 21:30:22.989850 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.989785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg7bk\" (UniqueName: \"kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk\") pod \"3366106e-cdc7-4b50-b8f3-7e144439fc75\" (UID: \"3366106e-cdc7-4b50-b8f3-7e144439fc75\") "
Sep 15 21:30:22.990075 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.990052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3366106e-cdc7-4b50-b8f3-7e144439fc75" (UID: "3366106e-cdc7-4b50-b8f3-7e144439fc75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:22.991893 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:22.991871    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk" (OuterVolumeSpecName: "kube-api-access-mg7bk") pod "3366106e-cdc7-4b50-b8f3-7e144439fc75" (UID: "3366106e-cdc7-4b50-b8f3-7e144439fc75"). InnerVolumeSpecName "kube-api-access-mg7bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:23.091370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:23.091292    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3366106e-cdc7-4b50-b8f3-7e144439fc75-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:23.091370 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:23.091319    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg7bk\" (UniqueName: \"kubernetes.io/projected/3366106e-cdc7-4b50-b8f3-7e144439fc75-kube-api-access-mg7bk\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:23.292972 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:23.292940    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:30:23.297077 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:23.297051    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2f34ef-69dfd8fc6b-9x7j5"]
Sep 15 21:30:24.690168 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:24.690129    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3366106e-cdc7-4b50-b8f3-7e144439fc75" path="/var/lib/kubelet/pods/3366106e-cdc7-4b50-b8f3-7e144439fc75/volumes"
Sep 15 21:30:24.690659 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:24.690605    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3583087f-08d1-4d7b-9aa2-a239b5a76d30" path="/var/lib/kubelet/pods/3583087f-08d1-4d7b-9aa2-a239b5a76d30/volumes"
Sep 15 21:30:29.574532 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.574499    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"]
Sep 15 21:30:29.574943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.574928    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3583087f-08d1-4d7b-9aa2-a239b5a76d30" containerName="revoke-keys"
Sep 15 21:30:29.574943 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.574944    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3583087f-08d1-4d7b-9aa2-a239b5a76d30" containerName="revoke-keys"
Sep 15 21:30:29.575042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.574963    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3366106e-cdc7-4b50-b8f3-7e144439fc75" containerName="payload-pre-processing"
Sep 15 21:30:29.575042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.574968    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3366106e-cdc7-4b50-b8f3-7e144439fc75" containerName="payload-pre-processing"
Sep 15 21:30:29.575042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.575022    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3366106e-cdc7-4b50-b8f3-7e144439fc75" containerName="payload-pre-processing"
Sep 15 21:30:29.575042 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.575034    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3583087f-08d1-4d7b-9aa2-a239b5a76d30" containerName="revoke-keys"
Sep 15 21:30:29.579531 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.579515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:29.582004 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.581982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:30:29.582135 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.582110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:30:29.586440 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.586416    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"]
Sep 15 21:30:29.758003 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.757970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt\" (UID: \"a73a6e48-dadd-4c47-b015-fb37d8a240e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:29.859054 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.858954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt\" (UID: \"a73a6e48-dadd-4c47-b015-fb37d8a240e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:29.859610 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.859592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt\" (UID: \"a73a6e48-dadd-4c47-b015-fb37d8a240e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:29.890892 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:29.890872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:30.162535 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:30.162440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"]
Sep 15 21:30:30.165001 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:30:30.164969    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73a6e48_dadd_4c47_b015_fb37d8a240e5.slice/crio-022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7 WatchSource:0}: Error finding container 022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7: Status 404 returned error can't find the container with id 022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7
Sep 15 21:30:31.023658 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:31.023619    2577 generic.go:356] "Generic (PLEG): container finished" podID="a73a6e48-dadd-4c47-b015-fb37d8a240e5" containerID="23a1be9a4547b44f5231fb0c326abb787eac216da73e2325d6f7954325167432" exitCode=0
Sep 15 21:30:31.024047 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:31.023740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt" event={"ID":"a73a6e48-dadd-4c47-b015-fb37d8a240e5","Type":"ContainerDied","Data":"23a1be9a4547b44f5231fb0c326abb787eac216da73e2325d6f7954325167432"}
Sep 15 21:30:31.024047 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:31.023792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt" event={"ID":"a73a6e48-dadd-4c47-b015-fb37d8a240e5","Type":"ContainerStarted","Data":"022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7"}
Sep 15 21:30:32.164088 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:32.164061    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:32.283916 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:32.283823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca\") pod \"a73a6e48-dadd-4c47-b015-fb37d8a240e5\" (UID: \"a73a6e48-dadd-4c47-b015-fb37d8a240e5\") "
Sep 15 21:30:32.284255 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:32.284228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a73a6e48-dadd-4c47-b015-fb37d8a240e5" (UID: "a73a6e48-dadd-4c47-b015-fb37d8a240e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:32.384786 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:32.384750    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73a6e48-dadd-4c47-b015-fb37d8a240e5-maas-api-service-ca\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:33.036872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:33.036839    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"
Sep 15 21:30:33.036872 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:33.036857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt" event={"ID":"a73a6e48-dadd-4c47-b015-fb37d8a240e5","Type":"ContainerDied","Data":"022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7"}
Sep 15 21:30:33.037101 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:33.036908    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="022fa8aad5965b4098c71108fd03587bdb31d3d2dfe1419dfb97ef5214cc27f7"
Sep 15 21:30:40.166126 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.166097    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"]
Sep 15 21:30:40.169275 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.169248    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-41ffdf-29825130-8pwbh"]
Sep 15 21:30:40.212722 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.212684    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:30:40.213218 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.213154    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" podUID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" containerName="payload-pre-processing" containerID="cri-o://d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b" gracePeriod=30
Sep 15 21:30:40.383252 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.383217    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"]
Sep 15 21:30:40.386410 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.386374    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-41ffdf-rjgvt"]
Sep 15 21:30:40.465745 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.465719    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:30:40.559048 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.559014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume\") pod \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") "
Sep 15 21:30:40.559262 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.559068    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99sdw\" (UniqueName: \"kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw\") pod \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\" (UID: \"bba393ea-9256-4fe7-8d60-a3e03e08e5ef\") "
Sep 15 21:30:40.559687 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.559655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bba393ea-9256-4fe7-8d60-a3e03e08e5ef" (UID: "bba393ea-9256-4fe7-8d60-a3e03e08e5ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:40.561300 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.561269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw" (OuterVolumeSpecName: "kube-api-access-99sdw") pod "bba393ea-9256-4fe7-8d60-a3e03e08e5ef" (UID: "bba393ea-9256-4fe7-8d60-a3e03e08e5ef"). InnerVolumeSpecName "kube-api-access-99sdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:40.660985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.660949    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-plugins-config-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:40.660985 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.660980    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99sdw\" (UniqueName: \"kubernetes.io/projected/bba393ea-9256-4fe7-8d60-a3e03e08e5ef-kube-api-access-99sdw\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:40.690931 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.690900    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93dabb3b-0a0e-4431-95c1-a55271d389e6" path="/var/lib/kubelet/pods/93dabb3b-0a0e-4431-95c1-a55271d389e6/volumes"
Sep 15 21:30:40.691274 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:40.691259    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73a6e48-dadd-4c47-b015-fb37d8a240e5" path="/var/lib/kubelet/pods/a73a6e48-dadd-4c47-b015-fb37d8a240e5/volumes"
Sep 15 21:30:41.081441 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.081407    2577 generic.go:356] "Generic (PLEG): container finished" podID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" containerID="d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b" exitCode=0
Sep 15 21:30:41.081669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.081447    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"
Sep 15 21:30:41.081669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.081489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" event={"ID":"bba393ea-9256-4fe7-8d60-a3e03e08e5ef","Type":"ContainerDied","Data":"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"}
Sep 15 21:30:41.081669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.081535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv" event={"ID":"bba393ea-9256-4fe7-8d60-a3e03e08e5ef","Type":"ContainerDied","Data":"90e91dc444833b9181df98012ec3dff79d1eacf92621950ad3695e4d7bf51966"}
Sep 15 21:30:41.081669 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.081557    2577 scope.go:117] "RemoveContainer" containerID="d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"
Sep 15 21:30:41.090534 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.090518    2577 scope.go:117] "RemoveContainer" containerID="d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"
Sep 15 21:30:41.090821 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:30:41.090803    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b\": container with ID starting with d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b not found: ID does not exist" containerID="d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"
Sep 15 21:30:41.090882 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.090831    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b"} err="failed to get container status \"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b\": rpc error: code = NotFound desc = could not find container \"d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b\": container with ID starting with d316328f3b5b95f645819633a4962dc237bf504444fec20b019c34f0b4c7c02b not found: ID does not exist"
Sep 15 21:30:41.106325 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.106303    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:30:41.113927 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:41.113906    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-41ffdf-5bcb44fdf8-qd6mv"]
Sep 15 21:30:42.690696 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:42.690661    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" path="/var/lib/kubelet/pods/bba393ea-9256-4fe7-8d60-a3e03e08e5ef/volumes"
Sep 15 21:30:47.508382 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:47.508348    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:30:47.508889 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:47.508627    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy" containerID="cri-o://fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c" gracePeriod=30
Sep 15 21:30:52.783494 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.783430    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:30:52.877744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877706    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.877744 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877798    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877871    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877912    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.877994    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqlv8\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket\") pod \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\" (UID: \"6d9d4cc2-8a28-4a34-b704-f8840fd2434b\") "
Sep 15 21:30:52.878391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:30:52.878391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878146    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:30:52.878391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878357    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.878391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:30:52.878391 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878377    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-workload-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.878666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:52.878666 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.878600    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:30:52.880473 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.880431    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:30:52.880651 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.880477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8" (OuterVolumeSpecName: "kube-api-access-nqlv8") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "kube-api-access-nqlv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:52.880743 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.880722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 21:30:52.880805 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.880747    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d9d4cc2-8a28-4a34-b704-f8840fd2434b" (UID: "6d9d4cc2-8a28-4a34-b704-f8840fd2434b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:52.979120 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979086    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqlv8\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-kube-api-access-nqlv8\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979120 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979113    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-credential-socket\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979120 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979122    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-podinfo\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979130    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-data\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979138    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-token\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979147    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istiod-ca-cert\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:52.979347 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:52.979156    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9d4cc2-8a28-4a34-b704-f8840fd2434b-istio-envoy\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:30:53.148225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.148144    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerID="fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c" exitCode=0
Sep 15 21:30:53.148225 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.148192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"
Sep 15 21:30:53.148402 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.148238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerDied","Data":"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"}
Sep 15 21:30:53.148402 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.148306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8" event={"ID":"6d9d4cc2-8a28-4a34-b704-f8840fd2434b","Type":"ContainerDied","Data":"950f3ee597ea79ca00a0994eaeca5158d6a23af69d2f68df4a40d17858ad2764"}
Sep 15 21:30:53.148402 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.148341    2577 scope.go:117] "RemoveContainer" containerID="fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"
Sep 15 21:30:53.157864 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.157853    2577 scope.go:117] "RemoveContainer" containerID="cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"
Sep 15 21:30:53.166795 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.166771    2577 scope.go:117] "RemoveContainer" containerID="fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"
Sep 15 21:30:53.167049 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:30:53.167031    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c\": container with ID starting with fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c not found: ID does not exist" containerID="fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"
Sep 15 21:30:53.167117 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.167056    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c"} err="failed to get container status \"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c\": rpc error: code = NotFound desc = could not find container \"fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c\": container with ID starting with fba4710379c8856396a8fec15055a7ef0e4775a425c4751664ad5652d8788b4c not found: ID does not exist"
Sep 15 21:30:53.167117 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.167070    2577 scope.go:117] "RemoveContainer" containerID="cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"
Sep 15 21:30:53.167343 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:30:53.167327    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229\": container with ID starting with cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229 not found: ID does not exist" containerID="cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"
Sep 15 21:30:53.167387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.167350    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229"} err="failed to get container status \"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229\": rpc error: code = NotFound desc = could not find container \"cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229\": container with ID starting with cb696955e05e2a381c93fe12ca3fffe22c9a5d138e7d1cfaf0a1093b86186229 not found: ID does not exist"
Sep 15 21:30:53.180614 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.180588    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:30:53.184941 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:53.184917    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-41ffdf-openshift-default-54976fc85b-55jd8"]
Sep 15 21:30:54.690831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:30:54.690802    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" path="/var/lib/kubelet/pods/6d9d4cc2-8a28-4a34-b704-f8840fd2434b/volumes"
Sep 15 21:33:12.820294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:12.820264    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:33:12.825598 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:12.825554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78t57_4dc6e8d9-fc44-4d1a-b425-d24bcccb9277/ovn-acl-logging/0.log"
Sep 15 21:33:26.875713 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:26.875677    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:33:26.876159 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:26.875879    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" podUID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" containerName="manager" containerID="cri-o://878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a" gracePeriod=10
Sep 15 21:33:29.337866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.337835    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:33:29.466839 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.466760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume\") pod \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") "
Sep 15 21:33:29.466990 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.466876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g9tt\" (UniqueName: \"kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt\") pod \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\" (UID: \"04b9f543-66fe-4f52-bb78-0319f0c2c48f\") "
Sep 15 21:33:29.467158 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.467121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04b9f543-66fe-4f52-bb78-0319f0c2c48f" (UID: "04b9f543-66fe-4f52-bb78-0319f0c2c48f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:33:29.469342 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.469314    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt" (OuterVolumeSpecName: "kube-api-access-8g9tt") pod "04b9f543-66fe-4f52-bb78-0319f0c2c48f" (UID: "04b9f543-66fe-4f52-bb78-0319f0c2c48f"). InnerVolumeSpecName "kube-api-access-8g9tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:33:29.568412 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.568385    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g9tt\" (UniqueName: \"kubernetes.io/projected/04b9f543-66fe-4f52-bb78-0319f0c2c48f-kube-api-access-8g9tt\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:33:29.568412 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.568410    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04b9f543-66fe-4f52-bb78-0319f0c2c48f-extensions-socket-volume\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:33:29.993102 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.993067    2577 generic.go:356] "Generic (PLEG): container finished" podID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" containerID="878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a" exitCode=0
Sep 15 21:33:29.993294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.993113    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"
Sep 15 21:33:29.993294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.993158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" event={"ID":"04b9f543-66fe-4f52-bb78-0319f0c2c48f","Type":"ContainerDied","Data":"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"}
Sep 15 21:33:29.993294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.993218    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2" event={"ID":"04b9f543-66fe-4f52-bb78-0319f0c2c48f","Type":"ContainerDied","Data":"f36c16a2546ebd57f376151b12102999e0619f5a159dcf16f5b8ebb0d992ae67"}
Sep 15 21:33:29.993294 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:29.993239    2577 scope.go:117] "RemoveContainer" containerID="878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"
Sep 15 21:33:30.003116 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:30.003088    2577 scope.go:117] "RemoveContainer" containerID="878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"
Sep 15 21:33:30.003400 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:33:30.003380    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a\": container with ID starting with 878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a not found: ID does not exist" containerID="878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"
Sep 15 21:33:30.003472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:30.003408    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a"} err="failed to get container status \"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a\": rpc error: code = NotFound desc = could not find container \"878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a\": container with ID starting with 878b5b38c9de0e328e65a2f9f013410a5e84571070aa7478f29b085c73cc2f8a not found: ID does not exist"
Sep 15 21:33:30.016093 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:30.016064    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:33:30.019522 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:30.019501    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f7rc2"]
Sep 15 21:33:30.690456 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:33:30.690412    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" path="/var/lib/kubelet/pods/04b9f543-66fe-4f52-bb78-0319f0c2c48f/volumes"
Sep 15 21:34:27.551878 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.551840    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"]
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552216    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" containerName="manager"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552228    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" containerName="manager"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552244    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" containerName="payload-pre-processing"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552250    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" containerName="payload-pre-processing"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552263    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552269    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552274    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552279    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552286    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73a6e48-dadd-4c47-b015-fb37d8a240e5" containerName="revoke-keys"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552291    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73a6e48-dadd-4c47-b015-fb37d8a240e5" containerName="revoke-keys"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552348    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552357    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bba393ea-9256-4fe7-8d60-a3e03e08e5ef" containerName="payload-pre-processing"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552363    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b9f543-66fe-4f52-bb78-0319f0c2c48f" containerName="manager"
Sep 15 21:34:27.552387 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.552369    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73a6e48-dadd-4c47-b015-fb37d8a240e5" containerName="revoke-keys"
Sep 15 21:34:27.555259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.555243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.557995 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.557974    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xslrl\""
Sep 15 21:34:27.568259 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.568237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"]
Sep 15 21:34:27.699008 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.698964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xwh2\" (UniqueName: \"kubernetes.io/projected/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-kube-api-access-5xwh2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.699198 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.699029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.800073 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.800035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwh2\" (UniqueName: \"kubernetes.io/projected/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-kube-api-access-5xwh2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.800224 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.800097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.800424 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.800409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.809186 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.809123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xwh2\" (UniqueName: \"kubernetes.io/projected/0c761e9e-0a1b-4a74-ad27-554ebbab05e5-kube-api-access-5xwh2\") pod \"kuadrant-operator-controller-manager-5769f99d9f-5x9gx\" (UID: \"0c761e9e-0a1b-4a74-ad27-554ebbab05e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:27.865428 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:27.865391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:28.159684 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:34:28.159647    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c761e9e_0a1b_4a74_ad27_554ebbab05e5.slice/crio-0f5afc960a3564fea8d60b9aa317b873ccd185f54eee855295e242b622bab53c WatchSource:0}: Error finding container 0f5afc960a3564fea8d60b9aa317b873ccd185f54eee855295e242b622bab53c: Status 404 returned error can't find the container with id 0f5afc960a3564fea8d60b9aa317b873ccd185f54eee855295e242b622bab53c
Sep 15 21:34:28.162427 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:28.162352    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"]
Sep 15 21:34:28.318925 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:28.318890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx" event={"ID":"0c761e9e-0a1b-4a74-ad27-554ebbab05e5","Type":"ContainerStarted","Data":"b4003051d68847de306569f15c0566e8b2252dc73979f2c368a396490d20c769"}
Sep 15 21:34:28.319131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:28.318978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx" event={"ID":"0c761e9e-0a1b-4a74-ad27-554ebbab05e5","Type":"ContainerStarted","Data":"0f5afc960a3564fea8d60b9aa317b873ccd185f54eee855295e242b622bab53c"}
Sep 15 21:34:28.319131 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:28.319007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:34:28.346838 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:28.346779    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx" podStartSLOduration=1.346756827 podStartE2EDuration="1.346756827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:34:28.346533004 +0000 UTC m=+2176.132901041" watchObservedRunningTime="2026-09-15 21:34:28.346756827 +0000 UTC m=+2176.133124836"
Sep 15 21:34:39.326379 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:34:39.326352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-5x9gx"
Sep 15 21:35:13.373758 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:13.373723    2577 scope.go:117] "RemoveContainer" containerID="10eeb99bd5cc47e0cfd1174d706bec11e0456d67833ec9afab69eff7a486adac"
Sep 15 21:35:13.382125 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:13.382102    2577 scope.go:117] "RemoveContainer" containerID="43f8b48eb7aa9486f44858d5a76ee391656da3bc0cb2de11775083c15417cea3"
Sep 15 21:35:28.873485 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.873450    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:35:28.874192 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.873936    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d9d4cc2-8a28-4a34-b704-f8840fd2434b" containerName="istio-proxy"
Sep 15 21:35:28.876930 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.876913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.880023 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.879992    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5e63750e-kserve-self-signed-certs\""
Sep 15 21:35:28.887279 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.887251    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:35:28.978101 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.978289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.978289 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.978388 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpbcl\" (UniqueName: \"kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.978388 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:28.978472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:28.978389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.079525 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpbcl\" (UniqueName: \"kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.079525 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.079833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.079833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.079833 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.080005 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.080005 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.079949    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.080205 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.080177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.080292 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.080185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.082245 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.082226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.082499 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.082482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.090107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.090077    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpbcl\" (UniqueName: \"kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl\") pod \"e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") " pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.188713 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.188627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:29.467006 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.466978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:35:29.469967 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:35:29.469935    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfab43066_fd2a_44e1_bd5e_e5fc2cbbc5ff.slice/crio-27c2ad24d8195fed6528cf3b3b39ac9c542676a200f31c86282150533ec82f5a WatchSource:0}: Error finding container 27c2ad24d8195fed6528cf3b3b39ac9c542676a200f31c86282150533ec82f5a: Status 404 returned error can't find the container with id 27c2ad24d8195fed6528cf3b3b39ac9c542676a200f31c86282150533ec82f5a
Sep 15 21:35:29.667955 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.667912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" event={"ID":"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff","Type":"ContainerStarted","Data":"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"}
Sep 15 21:35:29.668128 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.667966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" event={"ID":"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff","Type":"ContainerStarted","Data":"27c2ad24d8195fed6528cf3b3b39ac9c542676a200f31c86282150533ec82f5a"}
Sep 15 21:35:29.685913 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:29.685862    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" podStartSLOduration=1.685845509 podStartE2EDuration="1.685845509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:35:29.684887281 +0000 UTC m=+2237.471255289" watchObservedRunningTime="2026-09-15 21:35:29.685845509 +0000 UTC m=+2237.472213516"
Sep 15 21:35:39.189072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:39.188985    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:39.189072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:39.189037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:39.201793 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:39.201767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:39.732848 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:39.732815    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:35:54.187840 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:35:54.187810    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:36:09.208797 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.208678    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" podUID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" containerName="main" containerID="cri-o://6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6" gracePeriod=45
Sep 15 21:36:09.461076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.461016    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:36:09.568934 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.568896    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.568943    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.568980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpbcl\" (UniqueName: \"kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569129 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569089    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs\") pod \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\" (UID: \"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff\") "
Sep 15 21:36:09.569416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:09.569416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569311    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:09.569416 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569391    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home" (OuterVolumeSpecName: "home") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:09.569592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569515    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tmp-dir\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.569592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569530    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-model-cache\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.569592 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.569540    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-home\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.571500 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.571476    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl" (OuterVolumeSpecName: "kube-api-access-vpbcl") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "kube-api-access-vpbcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:36:09.571714 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.571686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm" (OuterVolumeSpecName: "dshm") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:09.571818 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.571734    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" (UID: "fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:36:09.670450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.670418    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-dshm\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.670450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.670447    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpbcl\" (UniqueName: \"kubernetes.io/projected/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-kube-api-access-vpbcl\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.670450 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.670458    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff-tls-certs\") on node \"ip-10-0-131-2.ec2.internal\" DevicePath \"\""
Sep 15 21:36:09.885401 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.885366    2577 generic.go:356] "Generic (PLEG): container finished" podID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" containerID="6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6" exitCode=0
Sep 15 21:36:09.885637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.885410    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"
Sep 15 21:36:09.885637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.885455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" event={"ID":"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff","Type":"ContainerDied","Data":"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"}
Sep 15 21:36:09.885637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.885504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt" event={"ID":"fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff","Type":"ContainerDied","Data":"27c2ad24d8195fed6528cf3b3b39ac9c542676a200f31c86282150533ec82f5a"}
Sep 15 21:36:09.885637 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.885526    2577 scope.go:117] "RemoveContainer" containerID="6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"
Sep 15 21:36:09.894877 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.894861    2577 scope.go:117] "RemoveContainer" containerID="6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"
Sep 15 21:36:09.895137 ip-10-0-131-2 kubenswrapper[2577]: E0915 21:36:09.895124    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6\": container with ID starting with 6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6 not found: ID does not exist" containerID="6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"
Sep 15 21:36:09.895188 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.895146    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6"} err="failed to get container status \"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6\": rpc error: code = NotFound desc = could not find container \"6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6\": container with ID starting with 6d7dacd4525a335f117f1fa338748c9e39eb8de1b17a113a8a11bd9b9ed745c6 not found: ID does not exist"
Sep 15 21:36:09.907934 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.907901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:36:09.917090 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:09.917062    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5e63750e-kserve-6949d48d7f-pfbvt"]
Sep 15 21:36:10.690621 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:10.690556    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" path="/var/lib/kubelet/pods/fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff/volumes"
Sep 15 21:36:13.405227 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:13.405195    2577 scope.go:117] "RemoveContainer" containerID="727e317ed5830a96034c3ab9ff2c76e4105ccd3f192a9433e22aa7cc033156ff"
Sep 15 21:36:13.413513 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:13.413493    2577 scope.go:117] "RemoveContainer" containerID="2e424f98cefff14b1e9c556e8d079b44f55c593f229ae7881c8d4f6203d69422"
Sep 15 21:36:13.421451 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:13.421427    2577 scope.go:117] "RemoveContainer" containerID="03f0f44bd35bb3a893aa24ec2fc279cdc9f108350ee80b36ab54ab742ef732ae"
Sep 15 21:36:13.429145 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:13.429119    2577 scope.go:117] "RemoveContainer" containerID="14dd36585e72d11be2d9051810601ba99be01cfefd052cb9edadd4a54520aa8e"
Sep 15 21:36:48.191831 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:48.191805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c6ckd_3962480d-e0db-4836-9890-3ec402d58dee/manager/0.log"
Sep 15 21:36:48.307993 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:48.307965    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hfcpz_2f654635-5148-406c-8c1f-9e9140abc361/manager/0.log"
Sep 15 21:36:48.542024 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:48.541997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sfsgb_05ef6d78-cfbd-4ce0-94c2-465be0e159c6/server/0.log"
Sep 15 21:36:48.657788 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:48.657749    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-246nq_489116ed-5fa3-4132-b563-b834a175f16e/manager/0.log"
Sep 15 21:36:49.003581 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:49.003533    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-wzdh8_86152b83-66b7-4871-b361-3bdec8959af9/copy-manifests/0.log"
Sep 15 21:36:49.017311 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:49.017277    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-wzdh8_86152b83-66b7-4871-b361-3bdec8959af9/manager/0.log"
Sep 15 21:36:49.588586 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:49.588507    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29825115-r6hvh_4e1fc651-2ac8-499d-98d5-d867b4963edb/cleanup/0.log"
Sep 15 21:36:49.707866 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:49.707837    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29825130-scbz9_ca474096-0847-4f4b-ae20-d08130f8b073/cleanup/0.log"
Sep 15 21:36:50.791011 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:50.790973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-6qw9x_ea87473d-8756-4cd4-bf5b-fdcc98bd30e6/manager/0.log"
Sep 15 21:36:51.013437 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:51.013408    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-5x9gx_0c761e9e-0a1b-4a74-ad27-554ebbab05e5/manager/0.log"
Sep 15 21:36:51.252462 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:51.252428    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-fkpmb_6730d897-7607-4a6f-872a-65c3f8104fdf/manager/0.log"
Sep 15 21:36:51.593998 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:51.593919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5_f9316c44-b627-454b-ad8a-e1c19d2687fa/istio-proxy/0.log"
Sep 15 21:36:51.713027 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:51.712978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nxgmr_0ef9e7d5-84be-493b-b871-923f429abd99/discovery/0.log"
Sep 15 21:36:51.928929 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:51.928851    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85f7c7997-x5n9t_4eb76639-bc49-4805-a0e4-7e27af5a8525/kube-auth-proxy/0.log"
Sep 15 21:36:52.038031 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:52.038000    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/1.log"
Sep 15 21:36:52.269180 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:52.269150    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"
Sep 15 21:36:52.914312 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:52.914280    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-6phq8_3a7631f7-401c-44d9-8ae3-374efb6b85f5/main/0.log"
Sep 15 21:36:53.131908 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:36:53.131884    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffknml_63b0bee9-dd9d-4e8f-951f-4e818ac04a3f/main/0.log"
Sep 15 21:37:01.518388 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:01.518332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p4qc5_723574f1-d85e-4b0e-9f37-9c0a96789c22/global-pull-secret-syncer/0.log"
Sep 15 21:37:01.720050 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:01.720020    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zp94c_bd315a5f-06a8-472b-8269-d87f8d05b72d/konnectivity-agent/0.log"
Sep 15 21:37:01.742779 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:01.742744    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-2.ec2.internal_4ed654b3d866e9d8cd5563c046da053f/haproxy/0.log"
Sep 15 21:37:07.020110 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:07.020066    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-6qw9x_ea87473d-8756-4cd4-bf5b-fdcc98bd30e6/manager/0.log"
Sep 15 21:37:07.093242 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:07.093208    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-5x9gx_0c761e9e-0a1b-4a74-ad27-554ebbab05e5/manager/0.log"
Sep 15 21:37:07.182785 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:07.182755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-fkpmb_6730d897-7607-4a6f-872a-65c3f8104fdf/manager/0.log"
Sep 15 21:37:09.068871 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.068838    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-d82l7_71185516-5e35-4581-8053-d7e65a26d39d/kube-state-metrics/0.log"
Sep 15 21:37:09.089306 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.089271    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-d82l7_71185516-5e35-4581-8053-d7e65a26d39d/kube-rbac-proxy-main/0.log"
Sep 15 21:37:09.114783 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.114751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-d82l7_71185516-5e35-4581-8053-d7e65a26d39d/kube-rbac-proxy-self/0.log"
Sep 15 21:37:09.412804 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.412723    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzm68_f4251f66-c059-45fb-900a-4e918abfc965/node-exporter/0.log"
Sep 15 21:37:09.435816 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.435790    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzm68_f4251f66-c059-45fb-900a-4e918abfc965/kube-rbac-proxy/0.log"
Sep 15 21:37:09.458506 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.458483    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzm68_f4251f66-c059-45fb-900a-4e918abfc965/init-textfile/0.log"
Sep 15 21:37:09.601777 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.601741    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/prometheus/0.log"
Sep 15 21:37:09.625472 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.625442    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/config-reloader/0.log"
Sep 15 21:37:09.648114 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.648087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/thanos-sidecar/0.log"
Sep 15 21:37:09.675395 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.675327    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/kube-rbac-proxy-web/0.log"
Sep 15 21:37:09.700112 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.700087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/kube-rbac-proxy/0.log"
Sep 15 21:37:09.733160 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.733135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/kube-rbac-proxy-thanos/0.log"
Sep 15 21:37:09.752682 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.752658    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19f9e565-a82b-4e3c-b9a8-57acb8a9757f/init-config-reloader/0.log"
Sep 15 21:37:09.837734 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.837700    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-qd4dh_d088c154-2616-46d2-8f30-428d5f16c35f/prometheus-operator-admission-webhook/0.log"
Sep 15 21:37:09.868998 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.868974    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5967c7fc7-9zkp9_8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d/telemeter-client/0.log"
Sep 15 21:37:09.895688 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.895657    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5967c7fc7-9zkp9_8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d/reload/0.log"
Sep 15 21:37:09.918353 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.918324    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5967c7fc7-9zkp9_8fb73f87-b5a3-44d0-a1d0-2bbc25b09d4d/kube-rbac-proxy/0.log"
Sep 15 21:37:09.975606 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.975494    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/thanos-query/0.log"
Sep 15 21:37:09.997307 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:09.997275    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/kube-rbac-proxy-web/0.log"
Sep 15 21:37:10.018355 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:10.018326    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/kube-rbac-proxy/0.log"
Sep 15 21:37:10.038977 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:10.038952    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/prom-label-proxy/0.log"
Sep 15 21:37:10.060099 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:10.060075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/kube-rbac-proxy-rules/0.log"
Sep 15 21:37:10.081403 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:10.081368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d59576f75-txgmp_9e93dcb8-84fa-4add-915c-37b307b0c7a5/kube-rbac-proxy-metrics/0.log"
Sep 15 21:37:11.618072 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.618032    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"]
Sep 15 21:37:11.618509 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.618493    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" containerName="main"
Sep 15 21:37:11.618603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.618511    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" containerName="main"
Sep 15 21:37:11.618653 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.618640    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fab43066-fd2a-44e1-bd5e-e5fc2cbbc5ff" containerName="main"
Sep 15 21:37:11.621919 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.621900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.624660 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.624633    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hwqzn\"/\"openshift-service-ca.crt\""
Sep 15 21:37:11.625781 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.625753    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hwqzn\"/\"default-dockercfg-dzqc9\""
Sep 15 21:37:11.625909 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.625765    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hwqzn\"/\"kube-root-ca.crt\""
Sep 15 21:37:11.634107 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.634081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"]
Sep 15 21:37:11.744868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.744824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-podres\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.744868 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.744868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-proc\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.745125 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.744893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v8nl\" (UniqueName: \"kubernetes.io/projected/7eaf4c3a-764d-465e-90bb-82be6808ab6e-kube-api-access-6v8nl\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.745125 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.744962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-lib-modules\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.745125 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.744995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-sys\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-podres\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-proc\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8nl\" (UniqueName: \"kubernetes.io/projected/7eaf4c3a-764d-465e-90bb-82be6808ab6e-kube-api-access-6v8nl\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-lib-modules\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-sys\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-proc\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-podres\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-sys\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.846433 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.846424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eaf4c3a-764d-465e-90bb-82be6808ab6e-lib-modules\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.855832 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.855805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8nl\" (UniqueName: \"kubernetes.io/projected/7eaf4c3a-764d-465e-90bb-82be6808ab6e-kube-api-access-6v8nl\") pod \"perf-node-gather-daemonset-8ltnk\" (UID: \"7eaf4c3a-764d-465e-90bb-82be6808ab6e\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:11.933182 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:11.933093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:12.208914 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:12.208889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"]
Sep 15 21:37:12.210112 ip-10-0-131-2 kubenswrapper[2577]: W0915 21:37:12.210085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7eaf4c3a_764d_465e_90bb_82be6808ab6e.slice/crio-b4681eba4e166a3473faad74c5b9c07b836d848f1e619ae098245f990f7b32fd WatchSource:0}: Error finding container b4681eba4e166a3473faad74c5b9c07b836d848f1e619ae098245f990f7b32fd: Status 404 returned error can't find the container with id b4681eba4e166a3473faad74c5b9c07b836d848f1e619ae098245f990f7b32fd
Sep 15 21:37:12.376235 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:12.376207    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68fc8ddc86-6xmbp_0cf7f666-7840-4ad0-9b4f-35ad16eea799/console/0.log"
Sep 15 21:37:13.208936 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:13.208900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk" event={"ID":"7eaf4c3a-764d-465e-90bb-82be6808ab6e","Type":"ContainerStarted","Data":"dc0e606524283e3824e7535086fbfeb8dcc17b2cc3771252426e89f6c1033cf1"}
Sep 15 21:37:13.209432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:13.208965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk" event={"ID":"7eaf4c3a-764d-465e-90bb-82be6808ab6e","Type":"ContainerStarted","Data":"b4681eba4e166a3473faad74c5b9c07b836d848f1e619ae098245f990f7b32fd"}
Sep 15 21:37:13.209432 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:13.209040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:13.225920 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:13.225867    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk" podStartSLOduration=2.225849093 podStartE2EDuration="2.225849093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 21:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:37:13.225047231 +0000 UTC m=+2341.011415239" watchObservedRunningTime="2026-09-15 21:37:13.225849093 +0000 UTC m=+2341.012217115"
Sep 15 21:37:13.484196 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:13.484164    2577 scope.go:117] "RemoveContainer" containerID="23a1be9a4547b44f5231fb0c326abb787eac216da73e2325d6f7954325167432"
Sep 15 21:37:14.115161 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:14.115130    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86bbp_b3dede8d-681d-4e4d-80c9-2539d00b566d/dns/0.log"
Sep 15 21:37:14.137721 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:14.137692    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86bbp_b3dede8d-681d-4e4d-80c9-2539d00b566d/kube-rbac-proxy/0.log"
Sep 15 21:37:14.314136 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:14.314102    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s7489_e67c6e70-a7b4-443e-b069-892373a430a1/dns-node-resolver/0.log"
Sep 15 21:37:14.911115 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:14.911081    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c684b_c2dd47b8-6d65-4982-9541-0e3293d59c85/node-ca/0.log"
Sep 15 21:37:15.860697 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:15.860663    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5_f9316c44-b627-454b-ad8a-e1c19d2687fa/istio-proxy/0.log"
Sep 15 21:37:16.243912 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:16.243883    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nxgmr_0ef9e7d5-84be-493b-b871-923f429abd99/discovery/0.log"
Sep 15 21:37:16.290134 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:16.290103    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85f7c7997-x5n9t_4eb76639-bc49-4805-a0e4-7e27af5a8525/kube-auth-proxy/0.log"
Sep 15 21:37:16.321316 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:16.321284    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/0.log"
Sep 15 21:37:16.324084 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:16.324061    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/1.log"
Sep 15 21:37:16.381559 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:16.381528    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"
Sep 15 21:37:17.352700 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:17.352644    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tfjhm_35b1de6c-9517-42bf-956d-32bca92ec8bd/serve-healthcheck-canary/0.log"
Sep 15 21:37:18.053248 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:18.053196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzwxd_edc5c4d7-d584-4d43-aff2-1afe67162ca6/kube-rbac-proxy/0.log"
Sep 15 21:37:18.082948 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:18.082918    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzwxd_edc5c4d7-d584-4d43-aff2-1afe67162ca6/exporter/0.log"
Sep 15 21:37:18.119900 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:18.119873    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzwxd_edc5c4d7-d584-4d43-aff2-1afe67162ca6/extractor/0.log"
Sep 15 21:37:19.225331 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:19.225301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-8ltnk"
Sep 15 21:37:20.514270 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:20.514228    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c6ckd_3962480d-e0db-4836-9890-3ec402d58dee/manager/0.log"
Sep 15 21:37:20.742558 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:20.742521    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hfcpz_2f654635-5148-406c-8c1f-9e9140abc361/manager/0.log"
Sep 15 21:37:20.810867 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:20.810797    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sfsgb_05ef6d78-cfbd-4ce0-94c2-465be0e159c6/server/0.log"
Sep 15 21:37:20.901076 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:20.901043    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-246nq_489116ed-5fa3-4132-b563-b834a175f16e/manager/0.log"
Sep 15 21:37:21.072460 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:21.072378    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-wzdh8_86152b83-66b7-4871-b361-3bdec8959af9/manager/0.log"
Sep 15 21:37:21.089708 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:21.089687    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-wzdh8_86152b83-66b7-4871-b361-3bdec8959af9/copy-manifests/0.log"
Sep 15 21:37:22.123772 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.123745    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5_f9316c44-b627-454b-ad8a-e1c19d2687fa/istio-proxy/0.log"
Sep 15 21:37:22.507839 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.507805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nxgmr_0ef9e7d5-84be-493b-b871-923f429abd99/discovery/0.log"
Sep 15 21:37:22.568136 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.568106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85f7c7997-x5n9t_4eb76639-bc49-4805-a0e4-7e27af5a8525/kube-auth-proxy/0.log"
Sep 15 21:37:22.617877 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.617846    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/0.log"
Sep 15 21:37:22.620269 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.620249    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/1.log"
Sep 15 21:37:22.754453 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:22.754421    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"
Sep 15 21:37:24.233869 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.233838    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cvm5t5_f9316c44-b627-454b-ad8a-e1c19d2687fa/istio-proxy/0.log"
Sep 15 21:37:24.623603 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.623462    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nxgmr_0ef9e7d5-84be-493b-b871-923f429abd99/discovery/0.log"
Sep 15 21:37:24.669471 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.669419    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85f7c7997-x5n9t_4eb76639-bc49-4805-a0e4-7e27af5a8525/kube-auth-proxy/0.log"
Sep 15 21:37:24.686805 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.686757    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/0.log"
Sep 15 21:37:24.690249 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.690227    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-q5rp9_0678c506-cf08-4e7b-a97d-ccda27c4803a/istio-proxy/1.log"
Sep 15 21:37:24.739060 ip-10-0-131-2 kubenswrapper[2577]: I0915 21:37:24.739027    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8mlr4_31959955-2c6a-40d1-9735-0d5a17b933a0/payload-processing/0.log"